lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <>
Subject Re: data import
Date Thu, 19 Mar 2015 20:27:48 GMT
On 3/19/2015 11:47 AM, abhishek tiwari wrote:
> <autoSoftCommit> <maxTime>500</maxTime> </autoSoftCommit>

You're doing soft commits as often as twice a second.  You have
configured 500 milliseconds here.  This might have something to do with
your slow indexing speed.  A soft commit is less expensive than a full
hard commit, but soft commits are *NOT* free, and they aren't even cheap.

I doubt that you *need* your documents to be visible within half a
second of indexing them ... and there's a good chance that even with
this config they won't be visible that soon, because each commit is
probably going to take longer than half a second to complete.  With a
500 millisecond autoSoftCommit configuration, your server may be doing
commit operations close to 100% of the time while indexing is happening.

Also, the dataimport handler is single threaded, so if you are only
using one handler definition in solrconfig.xml, there is no parallel
indexing.  You'll need to write your own multi-threaded indexing program
if you want parallel indexing.


View raw message