lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mark harwood <markharw...@yahoo.co.uk>
Subject Re: Suggested number of fields limit per Index
Date Thu, 03 Jan 2008 17:21:45 GMT
One thing to watch out for is the "norms" overhead which is one byte per field, per document.
These are byte arrays used in scoring to account for the length of fields in individual documents.
With hundreds of fields and millions of documents this can eat up memory.

The good news is you can opt to omit norms for a choice of field or if you have specify indexed
field as untokenized.

Cheers,
Mark

----- Original Message ----
From: "Dai, Chunhe" <Chunhe.Dai@fatwire.com>
To: java-user@lucene.apache.org
Sent: Thursday, 3 January, 2008 4:45:05 PM
Subject: Suggested number of fields limit per Index

I have been searching online could not find an exact answer; and
wondering if anyone here knows whether there is a preferred max number
of fields limit in lucene index?

We are in the process of deciding how our index would look like in our
lucene integration. For one of our approach, we could have a large
number of fields in the index - say maybe several hundred. But, each
Document in the index do not contain every of those fields and would
only have a few fields within those hundreds of fields (Probably in
tens). Does anyone ever have experience with set up like this? I am
wondering whether there is a potential performance issue with indexing
and searching.

Thanks.
Chunhe

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org






      ___________________________________________________________
Support the World Aids Awareness campaign this month with Yahoo! For Good http://uk.promotions.yahoo.com/forgood/

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message