lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Rosher" <>
Subject Re: Index updates between machines
Date Tue, 03 Apr 2007 14:51:41 GMT
Hi CW,

You might find this email from Doug Cutting useful, not NFS but using rsync
and hard links ... besides NFS without failover introduces a single point of


On 4/3/07, Chun Wei Ho <> wrote:
> We are running a search service on the internet using two machines. We
> have a crawler machine which crawls the web and merges new documents
> found into the Lucene index. We have a searcher machine which allows
> users to perform searches on the Lucene index.
> Periodically, we would copy the newest version of the index from the
> crawler machine over to the searcher machine (via copy over a NFS
> mount). The searcher would then detect the new version, close the old
> index, open the new index and resume the search service.
> As the index have been growing in size, we have been noticing that the
> search response time on the searcher machine increases drastically
> when an index (about 15GB) is being copied from the crawler to the
> searcher. Both machines run Fedora Core 4 and are on a gbps lan.
> We've tried a number of ways to reduce the impact of the copy over NFS
> on searching performance, such as "nice"ing the copy process, but to
> no avail. I wonder if anyone is running a lucene search service over a
> similar architecture and how you are managing the updates to the
> lucene index.
> Thanks!
> Regards,
> CW
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message