lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Briggs <acidbri...@gmail.com>
Subject Re: Suggested number of fields limit per Index
Date Thu, 03 Jan 2008 17:29:10 GMT
I'll give a quick opinion, and remember that is all it is.

Without more information of the types of documents your are storing, I
would say you are definitely going in the wrong direction.  In my
opinion, an index should describe the common attributes of all the
documents it contains.  You should not be designing an index with a
field commonality between documents of, in your case ~10%;  it should
be 100%.

Perhaps you should be creating multiple indexes that describe the
different types of documents (and I mean document in a lucene sense)?
Or perhaps you are being to fine grained within the content of your
fields?

But, from the sound of it, you have extremely disparate data that you
are trying to cram into a single index.  Like trying to index a car, a
lemon and a thought within the same index. I really don't like
bringing relational database comparisons in but this does sound like
you need to normalize that index and create some 'sub-indexes'.  Null
values, imho, are not good for an index.


Briggs.









On Jan 3, 2008 11:45 AM, Dai, Chunhe <Chunhe.Dai@fatwire.com> wrote:
> I have been searching online could not find an exact answer; and
> wondering if anyone here knows whether there is a preferred max number
> of fields limit in lucene index?
>
> We are in the process of deciding how our index would look like in our
> lucene integration. For one of our approach, we could have a large
> number of fields in the index - say maybe several hundred. But, each
> Document in the index do not contain every of those fields and would
> only have a few fields within those hundreds of fields (Probably in
> tens). Does anyone ever have experience with set up like this? I am
> wondering whether there is a potential performance issue with indexing
> and searching.
>
> Thanks.
> Chunhe
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>



-- 
"Conscious decisions by conscious minds are what make reality real"

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message