lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Muir <>
Subject Re: Anticipating a benchmark for direct posting format
Date Mon, 07 Apr 2014 13:30:33 GMT
On Mon, Apr 7, 2014 at 9:22 AM, Benson Margulies <> wrote:
> On Mon, Apr 7, 2014 at 9:14 AM, Robert Muir <> wrote:
>> On Thu, Apr 3, 2014 at 12:27 PM, Benson Margulies <> wrote:
>>> My takeaway from the prior conversation was that various people didn't
>>> entirely believe that I'd seen a dramatic improvement in query perfo
>>> using D-P-F, and so would not smile upon a patch intended to liberate
>>> D-P-F from codecs. It could be that the effect I saw has to do with
>>> the fact that our system depends on hitting and scoring 50% of the
>>> documents in an index with a lot of documents.
>> I dont understand the word "liberate" here. why is it such a problem
>> that this is a codec?
>  I don't want to have to declare my intentions at the time I create
> the index. I don't want to have to use D-P-F for all readers all the
> time. Because I want to be able to decide to open up an index with an
> arbitrary on-disk format and get the in-memory cache behavior of
> D-P-F. Thus 'liberate' -- split the question of 'keep a copy in
> memory' from the choice of the on-disk format.
>> i do not think we should give it any more status than that, it wastes
>> too much ram.
> It didn't seem like 'waste' when it solved a big practical for us. We
> had an application that was too slow, and had plenty of RAM available,
> and we were able to trade space for time by applying D-P-F.
> Maybe I'm going about this backwards; if I can come up with a small,
> inconspicuous proposed change that does what I want, there won't be
> any disagreement.

On the previous thread, i already mentioned that in such cases the
Directory API should be used.

Sorry, this DirectPostingsFormat is a huge trap. I don't think we need
_yet another_ way to waste ram, when someone can already do this via
directory (making the decision at any time).

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message