lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew Inger <mattin...@yahoo.com>
Subject High Availability
Date Mon, 04 Jan 2010 16:13:01 GMT
I'm kind of stuck and looking for suggestions for high availability options.  I've figured
out without much trouble how to get the master-slave replication working.  This eliminates
any single points of failure in the application in terms of the application's searching capability.

I would setup a master which would create the index and several slaves to act as the search
servers, and put them behind a load balancer to distribute the requests.  This would ensure
that if a slave node goes down, requests would continue to get serviced by the other nodes
that are still up.

The problem I have is that my particular application also has the capability to trigger index
updates from the user interface.  This means that the master now becomes a single point of
failure for the user interface.  

The basic idea of the app is that there are multiple oracle instances contributing to a single
document.  The volume and organization of the data (database links, normalization, etc...)
prevents any sort of fast querying via SQL to do querying of the documents.  The solution
is to build a lucene index (via solr), and use that for searching.  When updates are made
in the UI, we will also send the updates directly to the solr server as well (we don't want
to wait some arbitrary interval for a delta query to run).  

So you can see the problem here is that if the master is down, the sending of the updates
to the master solr server will fail, thus causing an application exception.

I have tried configuring multiple solr servers which are both setup as masters and slaves
to each other, but they keep clobber each other's index updates and rolling back each other's
delta updates.  It seems that the replication doesn't take the generation # into account and
check that the generation it's fetching is > the generation it already has before it applies
it.

I thought of maybe introducing a JMS queue to send my updates to and having the JMS message
listener set to manually acknowledge the messages only after a succesfull application of the
solrj api calls, but that seems kind of contrived, and is only a band-aid.

Does anyone have any suggestions?


 ----
mattinger@yahoo.com
"Once you start down the dark path, forever will it
dominate your destiny.  Consume you it will " - Yoda



      

Mime
View raw message