hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: Slow recovery on lost data node?
Date Thu, 09 Dec 2010 18:29:43 GMT
> Regarding using a lot of families... They are currently partitioned in a
> manner that reflects the various data groups that are likely to be read
> together... We're doing a lot of big scans on the regions of only one of
> those families, with scans of the full table being much shorter/rarer. By
> having separate store files I was hoping this separation would result in
> less overhead from not reading data that we simply don't need(stuff from the
> other families). Is the overhead from splitting the store files up large
> enough to make any savings on file access times not worth it? Or am I
> missing something else?

Well I'm missing a lot of information about your particular use case,
so there's no possible way for me to tell whether the overhead will be
bigger than if you used only one family in your specific case.

So what I can tell you is that in general more families is less
efficient in HBase. A region with 50 families works like 50 regions,
except that HBase doesn't handle that very well (for example, flush
size is calculated as the sum of all the families). What I mean is
that having 50 actual regions would be a lot better.

Also, when talking about "overhead", it's usually better to have some
data points to be able to compare solutions. Have you tried your few
use cases on a few different designs?  Any benchmarking?

Hope that helps,

J-D

Mime
View raw message