hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: region max filesize
Date Wed, 13 Jul 2011 03:04:27 GMT
Check out the copytables tool.  See if that will work for you. Here is
its usage:

stack@sv4borg227:~/hadoop-0.22$ ./bin/hadoop jar
~/hbase-0.91.0-SNAPSHOT/hbase-0.91.0-SNAPSHOT.jar copytable
Usage: CopyTable [--rs.class=CLASS] [--rs.impl=IMPL] [--starttime=X]
[--endtime=Y] [--new.name=NEW] [--peer.adr=ADR] <tablename>

 rs.class     hbase.regionserver.class of the peer cluster
              specify if different from current cluster
 rs.impl      hbase.regionserver.impl of the peer cluster
 starttime    beginning of the time range
              without endtime means from starttime to forever
 endtime      end of the time range
 new.name     new table's name
 peer.adr     Address of the peer cluster given in the format
 families     comma-seperated list of families to copy

 tablename    Name of the table to copy

 To copy 'TestTable' to a cluster that uses replication for a 1 hour window:
 $ bin/hbase org.apache.hadoop.hbase.mapreduce.CopyTable
--starttime=1265875194289 --endtime=1265878794289
--peer.adr=server1,server2,server3:2181:/hbase TestTable


On Tue, Jul 12, 2011 at 5:17 PM, 陈加俊 <cjjvictory@gmail.com> wrote:
> create simple m/r job to migrate data to new tables and drop the old ones。
> Could you put your code paste to here?
> On Wed, Jul 13, 2011 at 7:43 AM, Ravi Veeramachaneni <
> ravi.veeramachaneni@gmail.com> wrote:
>> Albert,
>> You doing partially right. The change will take into effect only to new
>> tables created going forward. There are two options I'm aware of, 1. Use
>> alter command on HBase shell to specify new region size. 2. Create new set
>> of tables and create simple m/r job to migrate data to new tables and drop
>> the old ones.
>> We followed the second approach when we bump the region size. The first
>> approach leaves the current region count But fills up until region gets to
>> 1GB. Our goal was to reduce number of regions to minimum, second approach
>> worked well for us.
>> Thanks,
>> Ravi
>> Sent from my mobile phone.
>> On Jul 11, 2011, at 8:56 PM, Albert Shau <ashau@yahoo-inc.com> wrote:
>> > Hi everyone,
>> >
>> > I'm using 0.90.3 and have the region max filesize set to 1gb in my
>> hbase-site.xml like so:
>> >
>> >  <property>
>> >    <name>hbase.hregion.max.filesize</name>
>> >    <value>1073741824</value>
>> >  </property>
>> >
>> > But when I look at the storefileSizeMB values listed on the regionserver
>> UI, they're all around 256mb, which makes me think the default value is
>> still being used for whatever reason.  I'm also setting other properties
>> like the handler count which are getting used so I'm pretty sure I have the
>> conf in the right place and it's getting read.  I've also tried restarting
>> hbase and doing a major compact on the table but the region sizes stay the
>> same.  Any ideas on what might be wrong?  Is there some way to verify the
>> max filesize is getting set correctly?
>> >
>> > Thanks,
>> > Albert
>> >
> --
> Thanks & Best regards
> jiajun

View raw message