hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: Question about HFile indices
Date Mon, 14 Jul 2014 20:36:15 GMT
We used to hold the entire HFile index in memory, although you could trade
off between the size of the index and amount of IO required to find the
desired records. Facebook found the aggregate size of indexes far too large
for their use case(s) so included the tiered index in the HFile V2
redesign. The design documents on HBASE-3857 might be worth a look.

On Mon, Jul 14, 2014 at 11:14 AM, Varun Sharma <varun@pinterest.com> wrote:

> Hi folks,
> I am wondering why we have a tiered index in the HFile format. Is it
> because the root index must fit in memory - hence must be limited in size.
> Does the bound on the root index pretty much dictate the index tiers ?
> Thank
> Varun

Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message