hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Liam Slusser <lslus...@gmail.com>
Subject Re: how to pre split a table whose row key is MD5(url)?
Date Tue, 13 May 2014 16:58:07 GMT
You can also create a table via the hbase shell with pre-split tables like
this...

Here is a 32-byte split into 16 different regions, using base16 (ie a md5
hash) for the key-type.

create 't1', {NAME => 'f1'},
{SPLITS=> ['10000000000000000000000000000000',
'20000000000000000000000000000000',
'30000000000000000000000000000000',
'40000000000000000000000000000000',
'50000000000000000000000000000000',
'60000000000000000000000000000000',
'70000000000000000000000000000000',
'80000000000000000000000000000000',
'90000000000000000000000000000000',
'a0000000000000000000000000000000',
'b0000000000000000000000000000000',
'c0000000000000000000000000000000',
'd0000000000000000000000000000000',
'e0000000000000000000000000000000',
'f0000000000000000000000000000000']}

thanks,
liam



On Tue, May 13, 2014 at 6:49 AM, sudhakara st <sudhakara.st@gmail.com>wrote:

> you can pre-splite table using you hex characters string for start key, end
> key and using  number of regions to spilit
>
>
> **************************************************************************************************************
> HTableDescriptor tableDes = new HTableDescriptor(tableName);
> tableDes.setValue(HTableDescriptor.SPLIT_POLICY,
> KeyPrefixRegionSplitPolicy.class.getName());
>
>            byte[][] splits =
> getHexSplits(SPLIT_START_KEY,SPLIT_END_KEY,NUM_OF_REGION_SPLIT);
>             admin.createTable(tableDes, splits);
>
>
> ******************************************************************************************************************
>  private  byte[][] getHexSplits(String startKey, String endKey, int
> numRegions) {
>         byte[][] splits = new byte[numRegions - 1][];
>         BigInteger lowestKey = new BigInteger(startKey, 8); //considering
> for first 8bytes to spilte
>         BigInteger highestKey = new BigInteger(endKey, 8);
>         BigInteger range = highestKey.subtract(lowestKey);
>         BigInteger regionIncrement =
> range.divide(BigInteger.valueOf(numRegions));
>         lowestKey = lowestKey.add(regionIncrement);
>         for (int i = 0; i < numRegions - 1; i++) {
>             BigInteger key =
> lowestKey.add(regionIncrement.multiply(BigInteger.valueOf(i)));
>             byte[] b = String.format("%016x", key).getBytes();
>             splits[i] = b;
>         }
>         return splits;
>     }
>
>
> *************************************************************************************************************
>
>
> On Mon, May 12, 2014 at 7:07 AM, Li Li <fancyerii@gmail.com> wrote:
>
> > thanks. I will try this.
> > by the way, byte range is -128 - 127
> >
> > On Mon, May 12, 2014 at 6:13 AM, Michael Segel
> > <michael_segel@hotmail.com> wrote:
> > > Simple answer… you really can’t.
> > > The best thing you can do is to pre split the table in to 4 regions
> > based on splitting the first byte in to 4 equal ranges.
> > (0-63,64-127,128-191,191-255)
> > >
> > > And hope that you’ll have an even split.
> > >
> > > In theory, over time you will.
> > >
> > >
> > > On May 8, 2014, at 1:58 PM, Li Li <fancyerii@gmail.com> wrote:
> > >
> > >> say I have 4 region server. How to pre split a table using MD5 as row
> > key?
> > >>
> > >
> >
>
>
>
> --
>
> Regards,
> ...sudhakara
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message