lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grant Ingersoll <gsing...@apache.org>
Subject Re: Suggested number of fields limit per Index
Date Thu, 03 Jan 2008 18:03:14 GMT
Another issues is how to generate queries.  If you have hundreds of  
fields, you may have to generate queries (e.g. using the  
MultfieldQueryParser) across all those fields just to find documents  
that _could_ have those fields.  This can lead to the dreaded  
TooManyClausesException.

That being said, Lucene can handle that many fields; I doubt, though,  
that many would consider it a best practice and I don't think there  
would be any indexing performance issues.  Number of fields can be a  
search issue, but I don't know what your requirements are to say for  
sure.

I would say that if you have alternative approaches that you think  
will work for your other requirements, and use less fields, then give  
that a try.  I don't know if I would go so far as say all fields  
should be in common, but that is a good thing to approach, as it makes  
things easier.  Are you sure you can't just map your fields into a  
common set?  Perhaps if you described the problem a bit more, we can  
help.

-Grant



On Jan 3, 2008, at 11:45 AM, Dai, Chunhe wrote:

> I have been searching online could not find an exact answer; and
> wondering if anyone here knows whether there is a preferred max number
> of fields limit in lucene index?
>
> We are in the process of deciding how our index would look like in our
> lucene integration. For one of our approach, we could have a large
> number of fields in the index - say maybe several hundred. But, each
> Document in the index do not contain every of those fields and would
> only have a few fields within those hundreds of fields (Probably in
> tens). Does anyone ever have experience with set up like this? I am
> wondering whether there is a potential performance issue with indexing
> and searching.
>
> Thanks.
> Chunhe
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>

--------------------------
Grant Ingersoll
http://lucene.grantingersoll.com
http://www.lucenebootcamp.com

Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ





---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message