hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Clint Morgan" <clint.mor...@troove.net>
Subject Re: Integer key range scan
Date Tue, 16 Dec 2008 23:21:11 GMT
Yeah, I had toyed with getting a format such that lexo ordering works but
this is rather tricky to do...

After we get user-specifiable per-table row-key comparators (Hbase 661).
Then this is much more straightforward, and you could do the same with idea
with doubles..

On Tue, Dec 16, 2008 at 3:11 PM, Jonathan Gray <jlist@streamy.com> wrote:

> Sorry, forgot to disclaim that this only works with unsigned integers.
>
> I assume based on the first post that this was for positive numbers?  A
> proper sort of signed numbers when storing as strings is not totally
> straightforward either as just adding a - to the front would actually have
> the first entry being the largest negative number (closest to 0) rather
> than
> the smallest number.  (ie. -1 is lexicographically before -100)
>
> Using intToBytes, negatives would be after all positives and would be in
> increasing order (more positive).
>
> It's feasible to design a reverse twos-complement encoding that would give
> proper ordering (leading 1 on positives instead of negatives) if you need
> support for signed integers and wanted to store binary.
>
> JG
>
> > -----Original Message-----
> > From: clint.a.m@gmail.com [mailto:clint.a.m@gmail.com] On Behalf Of
> > Clint Morgan
> > Sent: Tuesday, December 16, 2008 2:41 PM
> > To: hbase-user@hadoop.apache.org
> > Subject: Re: Integer key range scan
> >
> > Actually, thats not the case:
> >
> >  @Test
> >     public void test() {
> >
> >         Integer lastInt = 0;
> >         byte[] lastBytes = null;
> >         for (int i = -3; i < 3; i++) {
> >             byte[] current = Bytes.toBytes(i);
> >
> >             if (lastBytes != null) {
> >                 Assert.isTrue(lastInt.compareTo(i) < 0);
> >                 Assert.isTrue(Bytes.compareTo(lastBytes, current) < 0);
> >             }
> >             lastBytes = current;
> >             lastInt = i;
> >         }
> >     }
> >
> > Fails...
> >
> > On Tue, Dec 16, 2008 at 2:32 PM, Jonathan Gray <jlist@streamy.com>
> > wrote:
> >
> > > intToBytes is really just a "cast" to bytes, allowing you to store
> > the
> > > actual binary value of the integer.  It would be in increasing
> > numerical
> > > order.
> > >
> > > This is an example of the oddity that is Java...
> > >
> > > Great move by using big-endian in the virtual machine, but if you
> > want to
> > > actually access the binary the JVM is using for an integer you have
> > to
> > > write
> > > your own function that uses bitwise operations to extract the value
> > into a
> > > byte[].
> > >
> > > Anyways, the answer is Yes.
> > >
> > > JG
> > >
> > > > -----Original Message-----
> > > > From: clint.a.m@gmail.com [mailto:clint.a.m@gmail.com] On Behalf Of
> > > > Clint Morgan
> > > > Sent: Tuesday, December 16, 2008 2:13 PM
> > > > To: hbase-user@hadoop.apache.org
> > > > Subject: Re: Integer key range scan
> > > >
> > > > Does intToBytes preserve ordering? IE is the leicographic ordering
> > of
> > > > the
> > > > resulting byte [] the same as the ordering of the original ints?
> > > >
> > > > I think Edward needs that for his scan to work...
> > > >
> > > > -clint
> > > >
> > > > On Tue, Dec 16, 2008 at 1:25 PM, Jonathan Gray <jlist@streamy.com>
> > > > wrote:
> > > >
> > > > > I believe there are intToBytes/bytesToInt functions in
> > Bytes.java.
> > > > Rather
> > > > > than padding and storing as strings, you can just store as binary
> > > > numbers.
> > > > >  If you can't find those functions I can send you one of our
> > helper
> > > > classes
> > > > > that deals with binary data easily.
> > > > >
> > > > > Though this does make things a bit less pretty in web ui/shell
> > it's
> > > > far
> > > > > more efficient.
> > > > >
> > > > > JG
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: edward@udanax.org [mailto:edward@udanax.org] On Behalf
Of
> > > > Edward
> > > > > > J. Yoon
> > > > > > Sent: Tuesday, December 16, 2008 9:39 AM
> > > > > > To: hbase-user@hadoop.apache.org
> > > > > > Subject: Re: Integer key range scan
> > > > > >
> > > > > > Oh Yes, Thanks for tip J-D.
> > > > > >
> > > > > > On Wed, Dec 17, 2008 at 2:27 AM, Jean-Daniel Cryans
> > > > > > <jdcryans@apache.org> wrote:
> > > > > > > Yes, then you would scan with startrow = 0000000035 and
> > stoprow =
> > > > > > 0000000236
> > > > > > > if this is really what you want to do.
> > > > > > >
> > > > > > > J-D
> > > > > > >
> > > > > > > On Tue, Dec 16, 2008 at 12:25 PM, Edward J. Yoon
> > > > > > <edwardyoon@apache.org>wrote:
> > > > > > >
> > > > > > >> Like this? 0000000035 ~ 0000000235
> > > > > > >>
> > > > > > >> On Wed, Dec 17, 2008 at 2:21 AM, Jean-Daniel Cryans
> > > > > > <jdcryans@apache.org>
> > > > > > >> wrote:
> > > > > > >> > You can left pad with zeroes then use the scanning
> > facility
> > > > for
> > > > > > faster
> > > > > > >> > retrieval.
> > > > > > >> >
> > > > > > >> > J-D
> > > > > > >> >
> > > > > > >> > On Tue, Dec 16, 2008 at 12:18 PM, Edward J. Yoon
> > > > > > <edwardyoon@apache.org
> > > > > > >> >wrote:
> > > > > > >> >
> > > > > > >> >> I would use integer key and extract data from
range
> > scans.
> > > > But,
> > > > > > hbase
> > > > > > >> >> row/column is alphabetically sorted.
> > > > > > >> >>
> > > > > > >> >> So, I wrote like below:
> > > > > > >> >>
> > > > > > >> >> for (int i = 35; i <= 235; i++) {
> > > > > > >> >>  table.getRow(i, columns[]);
> > > > > > >> >> }
> > > > > > >> >>
> > > > > > >> >> It seems, causes too many requests over hbase.
> > > > > > >> >> Any suggestion?
> > > > > > >> >>
> > > > > > >> >> --
> > > > > > >> >> Best Regards, Edward J. Yoon @ NHN, corp.
> > > > > > >> >> edwardyoon@apache.org
> > > > > > >> >> http://blog.udanax.org
> > > > > > >> >>
> > > > > > >> >
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> --
> > > > > > >> Best Regards, Edward J. Yoon @ NHN, corp.
> > > > > > >> edwardyoon@apache.org
> > > > > > >> http://blog.udanax.org
> > > > > > >>
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Best Regards, Edward J. Yoon @ NHN, corp.
> > > > > > edwardyoon@apache.org
> > > > > > http://blog.udanax.org
> > > > >
> > > > >
> > >
> > >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message