lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andi Vajda <>
Subject Re: GData Server - Lucene storage
Date Fri, 02 Jun 2006 17:52:27 GMT

On Fri, 2 Jun 2006, jason rutherglen wrote:

> It might be interesting to merge using BDB into Solr, as an option to 
> provide better realtime updates.  Perhaps the replication could be used as 
> well in place of rsync?  I don't have any experience with BDB replication, 
> anyone have thoughts on the matter?

If you're thinking of using Berkeley DB as a the store behind the Lucene index 
via the DbDirectory Directory implementation, here are a few things to keep in 

   - always setUseCompoundFile(false)
     don't use compound lucene index files on top of Berkeley DB:
      . there is a bug that prevents this from working correctly
      . it makes no sense anyway since it duplicates what DbDirectory is
        already doing (all index files are stored in the same Berkeley DB file)
      . it slows things down

   - if you are using a transaction around all the index updates, you may want
     to consider doing all the index updates in a RAMDirectory first and then
     adding the RAMDirectory wholesale to the DbDirectory in that transaction.
     This makes indexing considerably faster (3 times for me) and does a LOT
     less thrashing around in Berkeley DB which can lead to a large number of
     transactional log files rapidly filling up your hard drive.

I'm not really sure if and how index merging works. For my use, having no 
merging is good enough since I never update existing documents, but always 
instead add a new version of them. The concept of version is tied to my 
application and each transaction corresponds to a new version.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message