lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <>
Subject Re: Working with a Distributed System
Date Fri, 01 Nov 2002 23:56:44 GMT
That is the approach I took at my previous job, which involved some
Lucene work.  I used sdist, to securely distribute the whole index (the
whole dir with index files) to a number of remote machines.

This may not work well if indices need to constantly be in sync, and if
the index can be modified on all index nodes.

How about using JMS and publish/subscribe with maybe time-stamped
messages, etc.?


--- Ype Kingma <> wrote:
> On Friday 01 November 2002 15:05, Rob Outar wrote:
> > All,
> >
> > 	I have what I think is an interesting problem.  I am working on a
> > distributed system where all repositories on each node have to be
> kept in
> > sync.  I am using Lucene on each node to index the data.  Users are
> allowed
> > to associate Fields with files, set values of existing fields,
> these fields
> > then have be also be associated with the same document on other
> nodes.  I
> > am using broadcast events to update the other nodes.  The problem
> is when a
> > new node joins in, I am not sure how to get the changes to the
> various
> > indexes to that node.  All nodes that are running together should
> be in
> > sync, but when a new node joins it does not know about any of the
> changes. 
> > The basic problem is how do I keep the indexes the same on all of
> the
> > nodes.  I though about maybe setting up a CVS Server and storing
> the index
> > in it then when a new node joins it checks out the index but I do
> not know
> > enough about the internal of Lucene to know if that will work, I
> will be
> > constantly committing files because the index will get updated a
> lot on the
> > various nodes, also will node b's committed files overwrite node
> a's files
> > which means nodes a changes to the index will be lost... very
> difficult
> > problem, if anyone has any thoughts on this subject I would love to
> hear
> > them.
> Assuming you run Unix, you might try and use rsync.
> It works like cp (copy) but it takes into account what is already on
> the 
> destination.
> See
> I'd like to hear how it works for lucene indexes...
> Kind regards,
> Ype
> --
> To unsubscribe, e-mail:  
> <>
> For additional commands, e-mail:
> <>

Do you Yahoo!?
HotJobs - Search new jobs daily now

To unsubscribe, e-mail:   <>
For additional commands, e-mail: <>

View raw message