lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <>
Subject Re: parallizing index building
Date Mon, 30 Jun 2003 17:23:44 GMT
Marc Dumontier wrote:
> I'm indexing 500 XML files each ~150Mb on an 8 CPU machine.
> I'm wondering what the best strategy for making maximum use of resources is. I have the
tweaked the single process indexer to index 5000 records (not files) in memory before writing
out to disk.
> Should i create an IndexThread and share the IndexWriter object across 5 threads..then
monitor when one ends to start another, etc. Or should i create difference indexes then to
a series of merges.

Creating multiple indexes in parallel and then merging them at the end 
will probably be fastest.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message