lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Muir <rcm...@gmail.com>
Subject Re: Anticipating a benchmark for direct posting format
Date Mon, 07 Apr 2014 13:30:33 GMT
On Mon, Apr 7, 2014 at 9:22 AM, Benson Margulies <bimargulies@gmail.com> wrote:
> On Mon, Apr 7, 2014 at 9:14 AM, Robert Muir <rcmuir@gmail.com> wrote:
>> On Thu, Apr 3, 2014 at 12:27 PM, Benson Margulies <bimargulies@gmail.com> wrote:
>>
>>>
>>> My takeaway from the prior conversation was that various people didn't
>>> entirely believe that I'd seen a dramatic improvement in query perfo
>>> using D-P-F, and so would not smile upon a patch intended to liberate
>>> D-P-F from codecs. It could be that the effect I saw has to do with
>>> the fact that our system depends on hitting and scoring 50% of the
>>> documents in an index with a lot of documents.
>>>
>>
>> I dont understand the word "liberate" here. why is it such a problem
>> that this is a codec?
>
>  I don't want to have to declare my intentions at the time I create
> the index. I don't want to have to use D-P-F for all readers all the
> time. Because I want to be able to decide to open up an index with an
> arbitrary on-disk format and get the in-memory cache behavior of
> D-P-F. Thus 'liberate' -- split the question of 'keep a copy in
> memory' from the choice of the on-disk format.
>
>
>>
>> i do not think we should give it any more status than that, it wastes
>> too much ram.
>
> It didn't seem like 'waste' when it solved a big practical for us. We
> had an application that was too slow, and had plenty of RAM available,
> and we were able to trade space for time by applying D-P-F.
>
> Maybe I'm going about this backwards; if I can come up with a small,
> inconspicuous proposed change that does what I want, there won't be
> any disagreement.
>
>

On the previous thread, i already mentioned that in such cases the
Directory API should be used.

Sorry, this DirectPostingsFormat is a huge trap. I don't think we need
_yet another_ way to waste ram, when someone can already do this via
directory (making the decision at any time).

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message