hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lars George <lars.geo...@gmail.com>
Subject Re: I can't get many versions of the specified column,but only get the latest version of the specified column
Date Thu, 24 Feb 2011 07:41:06 GMT
What error are you getting? The NPE?

As Tatsuya pointed out, you are using the same time stamps:

   private final long   ts2       = ts1 + 100;

   private final long   ts3       = ts1 + 100;

That cannot work, you are overriding cells.

Lars

On Thu, Feb 24, 2011 at 8:34 AM, 陈加俊 <cjjvictory@gmail.com> wrote:
> HTable object has not setAutoFlush. It's default value is true at my
> cluster.So I set it true as follows ,but error is still the same.
>
> public class GetRowVersionsTest extends TestCase
> {
>    private final byte[] family    = Bytes.toBytes("log");
>
>    private final byte[] qualifier = Bytes.toBytes("siteUrl");
>
>    private final byte[] rowKey    = Bytes.toBytes(1);
>
>    private final long   ts1       = 1298529542218L;
>
>    private final long   ts2       = ts1 + 100;
>
>    private final long   ts3       = ts1 + 100;
>
>    private final byte[] value1    = Bytes.toBytes("value1");
>
>    private final byte[] value2    = Bytes.toBytes("value2");
>
>    private final byte[] value3    = Bytes.toBytes("value3");
>
>    private void insert(final long ts, final byte[] value) throws
> IOException
>    {
>        final HTable table =
> IDMHBaseConfiguration.getTable(BigTableName.getSite());
>        table.setAutoFlush(false);
>        final Put put = new Put(rowKey);
>        put.add(family, qualifier, ts, value);
>        table.put(put);
>    }
>
>    private void sleep()
>    {
>        try
>        {
>            Thread.sleep(1000);
>        }
>        catch (final InterruptedException e)
>        {
>            e.printStackTrace();
>        }
>    }
>
>    @Test
>    public void testGetRowMultipleVersions() throws Exception
>    {
>        insert(ts1, value1);
>        sleep();
>        insert(ts2, value2);
>        sleep();
>        insert(ts3, value3);
>        sleep();
>
>        // check getRow with multiple versions
>        final HTable table =
> IDMHBaseConfiguration.getTable(BigTableName.getSite());
>        final Get get = new Get(rowKey);
>        get.setMaxVersions();
>        final Result r = table.get(get);
>
>        final List<KeyValue> list = r.list();
>        for (final KeyValue kv : list)
>        {
>            System.err.println(kv.getKey());
>            System.err.println(Bytes.toString(kv.getValue()));
>        }
>
>        final NavigableMap<byte[], NavigableMap<byte[], NavigableMap<Long,
> byte[]>>> map = r.getMap();
>        final NavigableMap<byte[], NavigableMap<Long, byte[]>> familyMap
=
> map.get(family);
>        final NavigableMap<Long, byte[]> versionMap =
> familyMap.get(qualifier);
>        for (final Map.Entry<Long, byte[]> entry : versionMap.entrySet())
>        {
>            System.err.println(entry.getKey());
>            System.err.println(Bytes.toString(entry.getValue()));
>        }
>        //        assertTrue(versionMap.size() == 3);
>
>        //        assertTrue(value1 == versionMap.get(ts1));
>        //        assertTrue(value2 == versionMap.get(ts2));
>        //        assertTrue(value3 == versionMap.get(ts3));
>
>        //        table.delete(new Delete(rowKey));
>        //        assertTrue(table.get(get).size() == 0);
>        //        table.close();
>    }
>
> On Thu, Feb 24, 2011 at 3:26 PM, Ryan Rawson <ryanobjc@gmail.com> wrote:
>
>> Does the HTable object have setAutoFlush(false) turned on by any chance?
>>
>> On Wed, Feb 23, 2011 at 11:22 PM, 陈加俊 <cjjvictory@gmail.com> wrote:
>> > line 89:        final NavigableMap<byte[], NavigableMap<Long, byte[]>>
>> > familyMap = map.get(family);
>> > map is null ,
>> > and strangely  I use r.list() instead,
>> > final List<KeyValue> list = r.list();
>> > r is null !
>> >
>> >
>> > 2011/2/24 Ryan Rawson <ryanobjc@gmail.com>
>> >>
>> >> Which line is line 89?
>> >>
>> >> Also it's preferable to do:
>> >> assertEquals(3, versionMap.size());
>> >> vs:
>> >> assertTrue(versionMap.size() == 3);
>> >>
>> >> since the error messages from the former are more descriptive
>> >> "expected 3 was 2".
>> >>
>> >> looking at the code it looks like it should work...
>> >>
>> >> On Wed, Feb 23, 2011 at 11:07 PM, 陈加俊 <cjjvictory@gmail.com>
wrote:
>> >> >  This is my test case ,but I get NPE some times .
>> >> >
>> >> > java.lang.NullPointerException
>> >> > at
>> >> >
>> >> >
>> com.uuwatch.idm.hbase.GetRowVersionsTest.testGetRowMultipleVersions(GetRowVersionsTest.java:89)
>> >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> >> > at
>> >> >
>> >> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> >> > at
>> >> >
>> >> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> >> > at java.lang.reflect.Method.invoke(Method.java:597)
>> >> > at
>> >> >
>> >> >
>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
>> >> > at
>> >> >
>> >> >
>> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>> >> > at
>> >> >
>> >> >
>> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
>> >> > at
>> >> >
>> >> >
>> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>> >> > at
>> >> >
>> >> >
>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
>> >> > at
>> >> >
>> >> >
>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
>> >> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
>> >> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
>> >> > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
>> >> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
>> >> > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
>> >> > at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
>> >> > at
>> >> >
>> >> >
>> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:49)
>> >> > at
>> >> >
>> >> >
>> org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
>> >> > at
>> >> >
>> >> >
>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:467)
>> >> > at
>> >> >
>> >> >
>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683)
>> >> > at
>> >> >
>> >> >
>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390)
>> >> > at
>> >> >
>> >> >
>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)
>> >> >
>> >> > public class GetRowVersionsTest extends TestCase
>> >> > {
>> >> >    private final byte[] family    = Bytes.toBytes("log");
>> >> >
>> >> >    private final byte[] qualifier = Bytes.toBytes("siteUrl");
>> >> >
>> >> >    private final byte[] rowKey    = Bytes.toBytes(1);
>> >> >
>> >> >    private final HTable table     =
>> >> > IDMHBaseConfiguration.getTable(BigTableName.getSite());
>> >> >
>> >> >    private final long   ts1       = 1298529542218L;
>> >> >
>> >> >    private final long   ts2       = ts1 + 100;
>> >> >
>> >> >    private final long   ts3       = ts1 + 100;
>> >> >
>> >> >    private final byte[] value1    = Bytes.toBytes("value1");
>> >> >
>> >> >    private final byte[] value2    = Bytes.toBytes("value2");
>> >> >
>> >> >    private final byte[] value3    = Bytes.toBytes("value3");
>> >> >
>> >> >    private void insert(final long ts, final byte[] value) throws
>> >> > IOException
>> >> >    {
>> >> >        final Put put = new Put(rowKey);
>> >> >        put.add(family, qualifier, ts, value);
>> >> >        table.put(put);
>> >> >    }
>> >> >
>> >> >    private void sleep()
>> >> >    {
>> >> >        try
>> >> >        {
>> >> >            Thread.sleep(1000);
>> >> >        }
>> >> >        catch (final InterruptedException e)
>> >> >        {
>> >> >            e.printStackTrace();
>> >> >        }
>> >> >    }
>> >> >
>> >> >    @Test
>> >> >    public void testGetRowMultipleVersions() throws Exception
>> >> >    {
>> >> >        insert(ts1, value1);
>> >> >        sleep();
>> >> >        insert(ts2, value2);
>> >> >        sleep();
>> >> >        insert(ts3, value3);
>> >> >        sleep();
>> >> >
>> >> >        // check getRow with multiple versions
>> >> >        final Get get = new Get(rowKey);
>> >> >        get.setMaxVersions();
>> >> >        final Result r = table.get(get);
>> >> >        final NavigableMap<byte[], NavigableMap<byte[],
>> >> > NavigableMap<Long,
>> >> > byte[]>>> map = r.getMap();
>> >> >        final NavigableMap<byte[], NavigableMap<Long, byte[]>>
>> familyMap
>> >> > =
>> >> > map.get(family);
>> >> >        final NavigableMap<Long, byte[]> versionMap =
>> >> > familyMap.get(qualifier);
>> >> >        for (final Map.Entry<Long, byte[]> entry :
>> versionMap.entrySet())
>> >> >        {
>> >> >            System.err.println(entry.getKey());
>> >> >            System.err.println(Bytes.toString(entry.getValue()));
>> >> >        }
>> >> >        //        assertTrue(versionMap.size() == 3);
>> >> >
>> >> >        //        assertTrue(value1 == versionMap.get(ts1));
>> >> >        //        assertTrue(value2 == versionMap.get(ts2));
>> >> >        //        assertTrue(value3 == versionMap.get(ts3));
>> >> >
>> >> >        //        table.delete(new Delete(rowKey));
>> >> >        //        assertTrue(table.get(get).size() == 0);
>> >> >        //        table.close();
>> >> >    }
>> >> > }
>> >> >
>> >> >
>> >> > On Thu, Feb 24, 2011 at 11:07 AM, Tatsuya Kawano
>> >> > <tatsuya6502@gmail.com>wrote:
>> >> >
>> >> >> Hi Jiajun,
>> >> >>
>> >> >> Make sure you don't have the same timestamp on every versions of
>> puts;
>> >> >> try
>> >> >> to put Thread.sleep() in your test(?) codes when necessary.
>> >> >>
>> >> >> You might not want to specify the timestamp by yourself but want
to
>> let
>> >> >> HBase to store appropriate ones.
>> >> >>
>> >> >> --
>> >> >> Tatsuya Kawano (Mr.)
>> >> >> Tokyo, Japan
>> >> >>
>> >> >>
>> >> >> On Feb 24, 2011, at 11:30 AM, Ryan Rawson <ryanobjc@gmail.com>
>> wrote:
>> >> >>
>> >> >> > There are test cases for this, the functionality DOES work,
>> something
>> >> >> > is
>> >> >> up...
>> >> >> >
>> >> >> > Without full code and full descriptions of your tables, debugging
>> is
>> >> >> > harder than it needs to be.  It's probably a simple typo
or
>> >> >> > something,
>> >> >> > check your code and table descriptions again. Many people
rely on
>> the
>> >> >> > multi version query capabilities and it is very unlikely to
be
>> broken
>> >> >> > in a released version of hbase.
>> >> >> >
>> >> >> > On Wed, Feb 23, 2011 at 6:27 PM, 陈加俊 <cjjvictory@gmail.com>
wrote:
>> >> >> >>  final List<KeyValue> list = result.list();
>> >> >> >>           for (final KeyValue it : list)
>> >> >> >>           {
>> >> >> >>               System.out.println(Bytes.toString(it.getKey()));
>> >> >> >>               System.out.println(Bytes.toString(it.getValue()));
>> >> >> >>           }
>> >> >> >> I can only get the last version!  why ? Is there any
testcase in
>> >> >> >> Hbase?
>> >> >> >>
>> >> >> >> On Thu, Feb 24, 2011 at 9:56 AM, 陈加俊 <cjjvictory@gmail.com>
>> wrote:
>> >> >> >>
>> >> >> >>> /**
>> >> >> >>>   * Create a sorted list of the KeyValue's in this
result.
>> >> >> >>>   *
>> >> >> >>>   * @return The sorted list of KeyValue's.
>> >> >> >>>   */
>> >> >> >>>  public List<KeyValue> list() {
>> >> >> >>>    if(this.kvs == null) {
>> >> >> >>>      readFields();
>> >> >> >>>    }
>> >> >> >>>    return isEmpty()? null: Arrays.asList(sorted());
>> >> >> >>>  }
>> >> >> >>>
>> >> >> >>> I will try it . Thank you very much!
>> >> >> >>>
>> >> >> >>> On Thu, Feb 24, 2011 at 9:45 AM, Buttler, David <
>> buttler1@llnl.gov>
>> >> >> wrote:
>> >> >> >>>
>> >> >> >>>> Result.list() ?
>> >> >> >>>> Putting the hbase source into your IDE of choice
(yay Eclipse!)
>> is
>> >> >> really
>> >> >> >>>> helpful
>> >> >> >>>>
>> >> >> >>>> Dave
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>> -----Original Message-----
>> >> >> >>>> From: 陈加俊 [mailto:cjjvictory@gmail.com]
>> >> >> >>>> Sent: Wednesday, February 23, 2011 5:42 PM
>> >> >> >>>> To: user@hbase.apache.org
>> >> >> >>>> Cc: Buttler, David
>> >> >> >>>> Subject: Re: I can't get many versions of the
specified
>> column,but
>> >> >> only
>> >> >> >>>> get the latest version of the specified column
>> >> >> >>>>
>> >> >> >>>> Thank you David !
>> >> >> >>>>
>> >> >> >>>> I alter the table schema as follow:
>> >> >> >>>>
>> >> >> >>>>> alter 'cjjIndexPageModify', {NAME => 'log'
, VERSIONS => 5 ,
>> >> >> >>>>> METHOD
>> >> >> =>
>> >> >> >>>> 'add'}
>> >> >> >>>>
>> >> >> >>>> How to iterate over KeyValues?  which method
in Result?
>> >> >> >>>>
>> >> >> >>>> On Thu, Feb 24, 2011 at 9:27 AM, Buttler, David
>> >> >> >>>> <buttler1@llnl.gov>
>> >> >> >>>> wrote:
>> >> >> >>>>
>> >> >> >>>>> What is your table schema set to?  By default
it holds 3
>> >> >> >>>>> versions.
>> >> >> >>>>> Also, you might iterating over KeyValues instead
of using the
>> Map
>> >> >> since
>> >> >> >>>> you
>> >> >> >>>>> don't really care about the organization,
just the time.
>> >> >> >>>>>
>> >> >> >>>>> Dave
>> >> >> >>>>>
>> >> >> >>>>> -----Original Message-----
>> >> >> >>>>> From: 陈加俊 [mailto:cjjvictory@gmail.com]
>> >> >> >>>>> Sent: Wednesday, February 23, 2011 5:22 PM
>> >> >> >>>>> To: user@hbase.apache.org
>> >> >> >>>>> Cc: Stack
>> >> >> >>>>> Subject: Re: I can't get many versions of
the specified
>> >> >> >>>>> column,but
>> >> >> only
>> >> >> >>>> get
>> >> >> >>>>> the latest version of the specified column
>> >> >> >>>>>
>> >> >> >>>>> I execute it five times at diffrent time.
>> >> >> >>>>>
>> >> >> >>>>> //put data by version
>> >> >> >>>>>
>> >> >> >>>>> final Put p = new Put(key); // key
>> >> >> >>>>> final long ts = System.currentTimeMillis();
>> >> >> >>>>>      p.add(FAMILY, q1, ts,v1);
>> >> >> >>>>>      p.add(FAMILY, q2, ts,v2);
>> >> >> >>>>>      p.add(FAMILY, q3, ts,v3);
>> >> >> >>>>>     table.put(p);
>> >> >> >>>>>
>> >> >> >>>>> So I can get five versions ,right?
>> >> >> >>>>>
>> >> >> >>>>> On Thu, Feb 24, 2011 at 2:06 AM, Stack <stack@duboce.net>
>> wrote:
>> >> >> >>>>>
>> >> >> >>>>>> What do you get for a result?
>> >> >> >>>>>>
>> >> >> >>>>>> You are only entering a single version
of each column, a
>> single
>> >> >> >>>>>> version of FAMILY:q1, a single version
FAMILY:q2, and a
>> >> >> >>>>>> FAMILY:q3.
>> >> >> >>>>>>
>> >> >> >>>>>> St.Ack
>> >> >> >>>>>>
>> >> >> >>>>>> On Wed, Feb 23, 2011 at 2:54 AM, 陈加俊
<cjjvictory@gmail.com>
>> >> >> >>>>>> wrote:
>> >> >> >>>>>>> I can't get many versions of the specified
column,but only
>> get
>> >> >> >>>>>>> the
>> >> >> >>>>> latest
>> >> >> >>>>>>> version of the specified column. Is
there anyone  help me?
>> >> >> >>>>>>>
>> >> >> >>>>>>> //put data by version
>> >> >> >>>>>>>
>> >> >> >>>>>>> final Put p = new Put(key); // key
>> >> >> >>>>>>> final long ts = System.currentTimeMillis();
>> >> >> >>>>>>>      p.add(FAMILY, q1, ts,v1);
>> >> >> >>>>>>>      p.add(FAMILY, q2, ts,v2);
>> >> >> >>>>>>>      p.add(FAMILY, q3, ts,v3);
>> >> >> >>>>>>>
>> >> >> >>>>>>>     table.put(p);
>> >> >> >>>>>>>
>> >> >> >>>>>>>
>> >> >> >>>>>>> //get all versions of data
>> >> >> >>>>>>>
>> >> >> >>>>>>> final Get get = new Get(key);
>> >> >> >>>>>>>          get.addFamily(FAMILY);
>> >> >> >>>>>>>          get.setMaxVersions(10);
>> >> >> >>>>>>>          final Result result
= htable.get(get);
>> >> >> >>>>>>>
>> >> >> >>>>>>>          final NavigableMap<byte[],
NavigableMap<byte[],
>> >> >> >>>>>>> NavigableMap<Long, byte[]>>>
map = result.getMap();
>> >> >> >>>>>>>          final Set<Entry<byte[],
NavigableMap<byte[],
>> >> >> >>>>>>> NavigableMap<Long, byte[]>>>>
entrySet = map.entrySet();
>> >> >> >>>>>>>          final Iterator<Entry<byte[],
NavigableMap<byte[],
>> >> >> >>>>>>> NavigableMap<Long, byte[]>>>>
iterator = entrySet
>> >> >> >>>>>>>                  .iterator();
>> >> >> >>>>>>>          while (iterator.hasNext())
>> >> >> >>>>>>>          {
>> >> >> >>>>>>>              final Entry<byte[],
NavigableMap<byte[],
>> >> >> >>>>>>> NavigableMap<Long, byte[]>>>
next = iterator.next();
>> >> >> >>>>>>>
>>  System.out.println(Bytes.toString(next.getKey()));
>> >> >> >>>>> //family
>> >> >> >>>>>>>              for (final Entry<byte[],
NavigableMap<Long,
>> >> >> >>>>>>> byte[]>>
>> >> >> >>>>>>> item : next.getValue().entrySet())
>> >> >> >>>>>>>              {
>> >> >> >>>>>>>
>> >> >> >>>>>>>  System.out.println(Bytes.toString(item.getKey()));
>> >> >> >>>>>>>                  for (final
Entry<Long, byte[]> _item :
>> >> >> >>>>>>> item.getValue().entrySet())
>> >> >> >>>>>>>                  {
>> >> >> >>>>>>>                      System.out.println(_item.getKey());
//q
>> >> >> >>>>>>>
>> >> >> >>>>>> System.out.println(Bytes.toString(_item.getValue()));
>> >> >> >>>>>>> //value
>> >> >> >>>>>>>                  }
>> >> >> >>>>>>>              }
>> >> >> >>>>>>>
>> >> >> >>>>>>>          }
>> >> >> >>>>>>>
>> >> >> >>>>>>> --
>> >> >> >>>>>>> Thanks & Best regards
>> >> >> >>>>>>> jiajun
>> >> >> >>>>>>>
>> >> >> >>>>>>
>> >> >> >>>>>
>> >> >> >>>>>
>> >> >> >>>>>
>> >> >> >>>>> --
>> >> >> >>>>> Thanks & Best regards
>> >> >> >>>>> jiajun
>> >> >> >>>>>
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>> --
>> >> >> >>>> Thanks & Best regards
>> >> >> >>>> jiajun
>> >> >> >>>>
>> >> >> >>>
>> >> >> >>>
>> >> >> >>>
>> >> >> >>> --
>> >> >> >>> Thanks & Best regards
>> >> >> >>> jiajun
>> >> >> >>>
>> >> >> >>>
>> >> >> >>
>> >> >> >>
>> >> >> >> --
>> >> >> >> Thanks & Best regards
>> >> >> >> jiajun
>> >> >> >>
>> >> >>
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Thanks & Best regards
>> >> > jiajun
>> >> >
>> >
>> >
>> >
>> > --
>> > Thanks & Best regards
>> > jiajun
>> >
>> >
>>
>
>
>
> --
> Thanks & Best regards
> jiajun
>

Mime
View raw message