hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jinsong Hu" <jinsong...@hotmail.com>
Subject Re: thrift for hbase in CDH3 broken ?
Date Fri, 03 Sep 2010 18:23:18 GMT
by the way, does anybody have a perl version of the test code ?

Jimmy

--------------------------------------------------
From: "Jinsong Hu" <jinsong_hu@hotmail.com>
Sent: Friday, September 03, 2010 11:17 AM
To: <user@hbase.apache.org>
Subject: Re: thrift for hbase in CDH3 broken ?

>
> I tried your code and indeed it works. but the java version doesn't work. 
> so it looks like it is a bug
> of the java library supplied by the thrift-0.2.0 version.
>
> Jimmy.
> --------------------------------------------------
> From: "Alexey Kovyrin" <alexey@kovyrin.net>
> Sent: Friday, September 03, 2010 12:31 AM
> To: <user@hbase.apache.org>
> Subject: Re: thrift for hbase in CDH3 broken ?
>
>> yes, Centos 5.5 + CDH3b2
>>
>> On Fri, Sep 3, 2010 at 3:26 AM, Jinsong Hu <jinsong_hu@hotmail.com> 
>> wrote:
>>> are you using CDH3 distribution ?
>>>
>>> Jinsong
>>>
>>>
>>> --------------------------------------------------
>>> From: "Alexey Kovyrin" <alexey@kovyrin.net>
>>> Sent: Friday, September 03, 2010 12:04 AM
>>> To: <user@hbase.apache.org>
>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>
>>>> http://github.com/kovyrin/hbase-thrift-client-examples - just wrote
>>>> this example and tested it in our cluster, works as expected.
>>>> For this to work you'd need to install rubygems and thrift gem (gem
>>>> install thrift).
>>>>
>>>> On Fri, Sep 3, 2010 at 12:01 AM, Jinsong Hu <jinsong_hu@hotmail.com>
>>>> wrote:
>>>>>
>>>>> Can you send me some ruby test code and so I can try against the 
>>>>> latest
>>>>> CDH3
>>>>> ?
>>>>>
>>>>> Jimmy.
>>>>>
>>>>> --------------------------------------------------
>>>>> From: "Alexey Kovyrin" <alexey@kovyrin.net>
>>>>> Sent: Thursday, September 02, 2010 8:15 PM
>>>>> To: <user@hbase.apache.org>
>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>
>>>>>> We use it in Scribd.com. All clients are ruby web apps.
>>>>>>
>>>>>> On Thu, Sep 2, 2010 at 10:49 PM, Todd Lipcon <todd@cloudera.com>

>>>>>> wrote:
>>>>>>>
>>>>>>> On Thu, Sep 2, 2010 at 5:35 PM, Jinsong Hu <jinsong_hu@hotmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Yes, I confirmed that it is indeed thrift server.
>>>>>>>>
>>>>>>>> and the fact that the API
>>>>>>>>
>>>>>>>>         List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>>>
>>>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>>>        {
>>>>>>>>>>            System.out.println(new String(name));
>>>>>>>>>>        }
>>>>>>>>>>
>>>>>>>>>
>>>>>>>> successfully printed all table names shows that it is indeed
thrift
>>>>>>>> server.
>>>>>>>>
>>>>>>>> if it is hue, it won't print the table names.
>>>>>>>>
>>>>>>>> Ah, sorry, I missed that in your original message. Not sure
what's 
>>>>>>>> up,
>>>>>>>> then
>>>>>>>
>>>>>>> - we don't have any changes in CDH that would affect this. Anyone

>>>>>>> here
>>>>>>> used
>>>>>>> thrift on 0.89.20100621?
>>>>>>>
>>>>>>> -Todd
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Jimmy.
>>>>>>>>
>>>>>>>> --------------------------------------------------
>>>>>>>> From: "Todd Lipcon" <todd@cloudera.com>
>>>>>>>> Sent: Thursday, September 02, 2010 5:18 PM
>>>>>>>>
>>>>>>>> To: <user@hbase.apache.org>
>>>>>>>> Subject: Re: thrift for hbase in CDH3 broken ?
>>>>>>>>
>>>>>>>>
>>>>>>>>  Hi Jinsong,
>>>>>>>>>
>>>>>>>>> Are you sure that the port you're connecting to is indeed
the 
>>>>>>>>> thrift
>>>>>>>>> server?
>>>>>>>>>
>>>>>>>>> Unfortunately both the HBase thrift server and the Hue
namenode
>>>>>>>>> plugin
>>>>>>>>> listen on port 9090, so you might be having an issue
where your 
>>>>>>>>> HBase
>>>>>>>>> client
>>>>>>>>> is trying to connect to the Namenode server instead of
HBase.
>>>>>>>>>
>>>>>>>>> You can verify the ports using a command like "/sbin/fuser
-n tcp
>>>>>>>>> 9090"
>>>>>>>>> to
>>>>>>>>> see which pid has it open, then cross reference against
sudo jps.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> -Todd
>>>>>>>>>
>>>>>>>>> On Thu, Sep 2, 2010 at 4:40 PM, Jinsong Hu 
>>>>>>>>> <jinsong_hu@hotmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>  Hi, There,
>>>>>>>>>>
>>>>>>>>>>  I am trying to test and see if thrift for hbase
works. I 
>>>>>>>>>> followed
>>>>>>>>>> the
>>>>>>>>>> example from
>>>>>>>>>>
>>>>>>>>>> http://www.workhabit.com/labs/centos-55-and-thriftscribe
>>>>>>>>>> http://incubator.apache.org/thrift/
>>>>>>>>>> http://wiki.apache.org/hadoop/Hbase/ThriftApi
>>>>>>>>>>
>>>>>>>>>> and wrote test code: I found that client.getTableNames();
>>>>>>>>>> returns all table names successfully, but the scanner
never 
>>>>>>>>>> returned
>>>>>>>>>> any record. and it even throws exception:
>>>>>>>>>>
>>>>>>>>>> org.apache.thrift.TApplicationException: get failed:
unknown 
>>>>>>>>>> result
>>>>>>>>>>     at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.recv_get(Hbase.java:785)
>>>>>>>>>>     at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.hadoop.hbase.thrift.generated.Hbase$Client.get(Hbase.java:750)
>>>>>>>>>>     at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.testThriftAPI(HbaseThriftTest.java:73)
>>>>>>>>>>     at
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> org.apache.hadoop.hbase.thrift.HbaseThriftTest.main(HbaseThriftTest.java:128)
>>>>>>>>>>
>>>>>>>>>> I checked the syntax and the table and confirmed
that the table 
>>>>>>>>>> does
>>>>>>>>>> have
>>>>>>>>>> record and it should
>>>>>>>>>> return result, but it doesn't. Can anybody tell me
what is wrong 
>>>>>>>>>> ?
>>>>>>>>>>
>>>>>>>>>> At this time, I am suspecting the thrift server shipped
with CDH3
>>>>>>>>>> may
>>>>>>>>>> have
>>>>>>>>>> problem. if there anyway
>>>>>>>>>> I can get this test code working ?
>>>>>>>>>>
>>>>>>>>>> I have to use thrift-0.2.0-incubating.tar.gz , as
I found that 
>>>>>>>>>> the
>>>>>>>>>> code
>>>>>>>>>> generated from hbase-thrift project
>>>>>>>>>> doesn't even compile with 0.3.0 and 0.4.0 thrift.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Jimmy.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>  public void testThriftAPI()
>>>>>>>>>>  {
>>>>>>>>>>    // Make socket
>>>>>>>>>>    TSocket transport = new TSocket("10.20.12.13",
9090);
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>    // Wrap in a protocol
>>>>>>>>>>    TBinaryProtocol protocol = new  TBinaryProtocol(transport);
>>>>>>>>>>
>>>>>>>>>>    Hbase.Client client = new Hbase.Client(protocol);
>>>>>>>>>>
>>>>>>>>>>    try {
>>>>>>>>>>        transport.open();
>>>>>>>>>>        List<byte[]> tableNamesList=client.getTableNames();
>>>>>>>>>>        for (byte [] name : tableNamesList)
>>>>>>>>>>        {
>>>>>>>>>>            System.out.println(new String(name));
>>>>>>>>>>        }
>>>>>>>>>>
>>>>>>>>>>        String tableName="HEARTBEAT_CLUSTER";
>>>>>>>>>>        String startRow="";
>>>>>>>>>>        List<byte[]> columns= new ArrayList<byte[]>();
>>>>>>>>>>        columns.add("fields:time_format".getBytes());
>>>>>>>>>>        columns.add("fields:customer_id".getBytes());
>>>>>>>>>>
>>>>>>>>>>        int aScannerId=0;
>>>>>>>>>>        try {
>>>>>>>>>>
>>>>>>>>>>            TCell cell=client.get(tableName.getBytes(),
>>>>>>>>>> "2010-08\tproduction-2".getBytes(),
>>>>>>>>>> "fields:customer_id".getBytes());
>>>>>>>>>>            if (cell != null)
>>>>>>>>>>            {
>>>>>>>>>>               String value= new String( cell.value);
>>>>>>>>>>               System.out.println(value);
>>>>>>>>>>            }
>>>>>>>>>>            aScannerId=client.scannerOpen(tableName.getBytes(),
>>>>>>>>>> startRow.getBytes(), columns);
>>>>>>>>>>            TRowResult result=null;
>>>>>>>>>>            for (int i=0; i< 10; i++)
>>>>>>>>>>            {
>>>>>>>>>>                try {
>>>>>>>>>>                result=client.scannerGet(aScannerId);
>>>>>>>>>>                } catch (Exception ex1)
>>>>>>>>>>                {
>>>>>>>>>>
>>>>>>>>>>                }
>>>>>>>>>>                if (result ==null) break;
>>>>>>>>>>
>>>>>>>>>>                String 
>>>>>>>>>> timeFormat=result.getFieldValue(0).toString();
>>>>>>>>>>                String 
>>>>>>>>>> customerId=result.getFieldValue(1).toString();
>>>>>>>>>>                System.out.println(timeFormat + "\t"
+ 
>>>>>>>>>> customerId);
>>>>>>>>>>            }
>>>>>>>>>>        } catch (Exception ex)
>>>>>>>>>>        {
>>>>>>>>>>            ex.printStackTrace();
>>>>>>>>>>        }
>>>>>>>>>>        finally {
>>>>>>>>>>            if (aScannerId >0 ) client.scannerClose(aScannerId);
>>>>>>>>>>        }
>>>>>>>>>>    } catch (Exception exp)
>>>>>>>>>>    {
>>>>>>>>>>        exp.printStackTrace();
>>>>>>>>>>    }
>>>>>>>>>>    finally{
>>>>>>>>>>        transport.close();
>>>>>>>>>>    }
>>>>>>>>>>
>>>>>>>>>>  }
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Todd Lipcon
>>>>>>>>> Software Engineer, Cloudera
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Todd Lipcon
>>>>>>> Software Engineer, Cloudera
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Alexey Kovyrin
>>>>>> http://kovyrin.net/
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Alexey Kovyrin
>>>> http://kovyrin.net/
>>>>
>>>
>>
>>
>>
>> -- 
>> Alexey Kovyrin
>> http://kovyrin.net/
>>
> 

Mime
View raw message