hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sudhakara st <sudhakara...@gmail.com>
Subject Re: how to pre split a table whose row key is MD5(url)?
Date Tue, 13 May 2014 13:49:18 GMT
you can pre-splite table using you hex characters string for start key, end
key and using  number of regions to spilit

**************************************************************************************************************
HTableDescriptor tableDes = new HTableDescriptor(tableName);
tableDes.setValue(HTableDescriptor.SPLIT_POLICY,
KeyPrefixRegionSplitPolicy.class.getName());

           byte[][] splits =
getHexSplits(SPLIT_START_KEY,SPLIT_END_KEY,NUM_OF_REGION_SPLIT);
            admin.createTable(tableDes, splits);

******************************************************************************************************************
 private  byte[][] getHexSplits(String startKey, String endKey, int
numRegions) {
        byte[][] splits = new byte[numRegions - 1][];
        BigInteger lowestKey = new BigInteger(startKey, 8); //considering
for first 8bytes to spilte
        BigInteger highestKey = new BigInteger(endKey, 8);
        BigInteger range = highestKey.subtract(lowestKey);
        BigInteger regionIncrement =
range.divide(BigInteger.valueOf(numRegions));
        lowestKey = lowestKey.add(regionIncrement);
        for (int i = 0; i < numRegions - 1; i++) {
            BigInteger key =
lowestKey.add(regionIncrement.multiply(BigInteger.valueOf(i)));
            byte[] b = String.format("%016x", key).getBytes();
            splits[i] = b;
        }
        return splits;
    }

*************************************************************************************************************


On Mon, May 12, 2014 at 7:07 AM, Li Li <fancyerii@gmail.com> wrote:

> thanks. I will try this.
> by the way, byte range is -128 - 127
>
> On Mon, May 12, 2014 at 6:13 AM, Michael Segel
> <michael_segel@hotmail.com> wrote:
> > Simple answer… you really can’t.
> > The best thing you can do is to pre split the table in to 4 regions
> based on splitting the first byte in to 4 equal ranges.
> (0-63,64-127,128-191,191-255)
> >
> > And hope that you’ll have an even split.
> >
> > In theory, over time you will.
> >
> >
> > On May 8, 2014, at 1:58 PM, Li Li <fancyerii@gmail.com> wrote:
> >
> >> say I have 4 region server. How to pre split a table using MD5 as row
> key?
> >>
> >
>



-- 

Regards,
...sudhakara

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message