lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yonik Seeley <>
Subject Re: scalability w/ number of fields
Date Tue, 05 Apr 2005 14:42:25 GMT
Optimize performance update (with tons of indexed fields):

We had a timing bug... ignore the hour I first reported.  Here are the
current numbers:

indexed_fields=6791  index_size=3.9GB  optimize_time=21min
indexed_fields=3216  index_size=2.0GB  optimize_time=9min
indexed_fields=2080  index_size=1.4GB  optimize_time=4min

It's a little apples-to-oranges since we simply removed some of the
fields to test a lower field count (and hence the index size also goes


On Apr 4, 2005 5:35 PM, Yonik Seeley <> wrote:
> I know Lucene is very scalable in many ways, but how about number of fieldnames?
> We have an index using around 6000 unique fieldnames,
> 450,000 documents, and a total index size of 4GB.   It's very
> sparse... documents don't have that many fields, but the number of
> different fieldtypes is huge.
> An optimize of this index took about an hour (mergefactor 10, compound index)
> This is on enterprise hardware (fast SCSI raid, 6GB RAM, dual 2.8GHz Xeon).
> The JVM was Java5 with 2.5GB heap.
> This seems very long... anyone have any insights?
> We'll be running more tests to see if decreasing the number of fields
> has an impact.
> -Yonik

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message