lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fuad Efendi" <f...@efendi.ca>
Subject NativeFSLockFactory, ConcurrentMergeScheduler: why locks?
Date Tue, 11 Aug 2009 17:08:44 GMT
1.       I always have files lucene-xxxx-write.lock and
lucene-xxxx-n-write.lock which I believe shouldn't be used with
NativeFSLockFactory

2.       I use mergeFactor=100 and ramBufferSizeMB=256, few GB indes size. I
tried mergeFactor=10 and mergeFactor=1000.

 

 

It seems ConcurrentMergeScheduler locks everything instead of using separate
thread on background...

 

 

So that my configured system spents half an hour to UPDATE (probably
existing in the index) million of documents, then it stops and waits few
hours for index merge which is extremely slow (a lot of deletes?)

 

With mergeFactor=1000 I had extremely performant index updates (50,000,000 a
first day), and then I was waiting more than 2 days when merge complete (and
was forced to kill process).

 

Why it locks everything?

 

Thanks,

Fuad  

 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message