lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrzej Bialecki>
Subject Re: Does lucene support distributed indexing?
Date Tue, 29 Apr 2008 11:44:18 GMT
Vaijanath N. Rao wrote:
> Hi all,
> How about adding hadoop support for distributed indexing. If required I 
> can start working on this. If Hadoop is the fesiable option.
> Also what other technique one can think for doing distributed Indexing. 
> Currently I am planning on extending the SolrJ to keep a map of where 
> the document has gone and trying to get a distibuted Indexing.

DistributedFileSystem performance for random seeks is several times 
worse than that of LocalFileSystem. This directly impacts Lucene 
response time.

One solution would be to implement the searching as an application that 
executes in a distributed fashion (not sure if map-reduce is the best 
model here), but first copies the indexes to LocalFileSystem.

Best regards,
Andrzej Bialecki     <><
  ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration  Contact: info at sigram dot com

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message