hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cheng Su <scarcer...@gmail.com>
Subject Re: Does hbase.hregion.max.filesize have a limit?
Date Fri, 02 Nov 2012 01:20:08 GMT
Thank you all guys.

I found out that I misunderstood the "size of a region" and "size of a
region server".
I found this property
193-  <property>
194-    <name>hbase.regionserver.regionSplitLimit</name>
195-    <value>2147483647</value>
196-    <description>Limit for the number of regions after which no more region
197:    splitting should take place. This is not a hard limit for the number of
198:    regions but acts as a guideline for the regionserver to stop
splitting after
199:    a certain limit. Default is set to MAX_INT; i.e. do not block splitting.
200-    </description>
201-  </property>

So in practice, a region server can handle enough regions, so I don't
need worry about the store size.

Thank you all again.

On Fri, Nov 2, 2012 at 12:39 AM, Jeremy Carroll <phobos182@gmail.com> wrote:
> To speak to 'if it's possible', yes it is. We have some tables over here at
> Klout during testing where we set the max region size to 100Gb, and
> actually had tables of that size during a MR job that created HFileV2's for
> us to import. So I can say that I have seen 100Gb regions that still work.
>
> As to if this is a good idea, it's probably not. As a capacity planning
> exercise we added additional nodes to the cluster, and split these regions
> down to 10-20Gb in size.
>
> On Wed, Oct 31, 2012 at 11:45 PM, Cheng Su <scarcer.cn@gmail.com> wrote:
>
>> Does that means the max file size of 1 cf is 20G? If I have 3 region
>> servers, then 60G total?
>> I have a very large table, size of one cf (contains only one column)
>> may exceed 60G.
>> Is there any chance to store the data without increase machines?
>>
>> Can multiple region servers runs on one real machine?
>> (I guess not though)
>>
>> On Thu, Nov 1, 2012 at 1:35 PM, lars hofhansl <lhofhansl@yahoo.com> wrote:
>> > The tribal knowledge would say about 20G is the max.
>> > The fellas from Facebook will have a more definite answer.
>> >
>> > -- Lars
>> >
>> >
>> >
>> > ________________________________
>> >  From: Cheng Su <scarcer.cn@gmail.com>
>> > To: user@hbase.apache.org
>> > Sent: Wednesday, October 31, 2012 10:22 PM
>> > Subject: Does hbase.hregion.max.filesize have a limit?
>> >
>> > Hi, all.
>> >
>> > I have a simple question: does hbase.hregion.max.filesize have a limit?
>> > May I specify a very large value to this? like 40G or more? (don't
>> > consider the performance)
>> > I didn't find any description about this from official site or google.
>> >
>> > Thanks.
>> >
>> > --
>> >
>> > Regards,
>> > Cheng Su
>>
>>
>>
>> --
>>
>> Regards,
>> Cheng Su
>>



-- 

Regards,
Cheng Su

Mime
View raw message