lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karsten F." <>
Subject Re: Per user data store
Date Wed, 06 Aug 2008 08:31:03 GMT


I want to agree with the advice of using only one index.

And I want to add two reasons:
1. Sorting and caching are working with the lucene-document-numbers.
In case of lucene "warming up" means that a lot of int-Arrays and bitsets
are stored in main memory.
If you using different MultiReader for each user all caching is also

2. you should think about what happened, if you get new users:
Most possible you will get a user "with the same permissions as XY".
So you don't want to copy a index-file or insert a new value in an existing
But you can easly copy the filter of an existing user.
(Which also means that I suggest not to use a field "userids with
read-permission". It is better to decouple userids and index).

But this reasons are only good for my thinking of amount of users, ratio of
deleting and adding documents and period of valid documents. 

So I again agree with Erick, that you should tell more about your use case.

Best regards


Erick Erickson wrote:
> I'd start out with one index, if for no other reason
> than keeping track of one index for each user would
> be a royal pain in the neck. You haven't told us
> how many users or documents you expect,
> so that's just a guess. There's one answer perhaps
> if you wind up with a 10M index, another if it's 10T.....
> Filtering on the username is a fine idea, although
> I'd also start by just ANDing in the username to
> the query to start. Then measure your resonse
> time. Note that the first time you open a reader, the
> response will be slow so measure queries 2-n
> instead.
> I don't know the guts of Lucene, but my indexes do NOT
> grow linearly with the data. After a very few docs, adding,
> say, 1M of data does not cause the data to grow by 1M (or
> even close to that) for fields that are NOT stored. I've
> learned to just trust that the very bright people who work
> on Lucene have "done the right thing" <G>...
> Best
> Erick

View this message in context:
Sent from the Lucene - Java Users mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message