lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Trejkaz <>
Subject Re: OutOfMemoryError indexing large documents
Date Thu, 27 Nov 2014 01:34:14 GMT
On Wed, Nov 26, 2014 at 2:09 PM, Erick Erickson <> wrote:
> Well
> 2> seriously consider the utility of indexing a 100+M file. Assuming
> it's mostly text, lots and lots and lots of queries will match it, and
> it'll score pretty low due to length normalization. And you probably
> can't return it to the user. And highlighting it will be a performance
> problem. And may blow out memory too. And...

Meanwhile, some of our users have expressed concern that they can't
view a 2GB text file which was returned in a Lucene result. They even
want to see the term hits and expect that to somehow perform the same
as a small file. Totally unreasonable. :)


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message