hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bruce Williams" <williams.br...@gmail.com>
Subject Re: Thrift API vs Java API
Date Mon, 15 Dec 2008 14:13:37 GMT
As far as it being a patch to Thrift, in all the examples you get a
socket from Thrift and buffer it yourself.

Typical code:

# Make socket
transport = TSocket.TSocket('localhost', 9090)

# Buffering is critical. Raw sockets are very slow
transport = TTransport.TBufferedTransport(transport)

# Wrap in a protocol
protocol = TBinaryProtocol.TBinaryProtocol(transport)

Bruce

On Mon, Dec 15, 2008 at 6:07 AM, Slava Gorelik <slava.gorelik@gmail.com> wrote:
> Well.
> I changed TSocket to work with BufferedStream and result is surprising fast,
> it has same performance and some times even better than HBase Java API and
> this sounds weird.
>
> How could it be ?
>
> Best Regards.
>
> On Sun, Dec 14, 2008 at 11:42 PM, stack <stack@duboce.net> wrote:
>
>> Erik, is that a patch to Thrift?  Or just a configuration?
>> Thanks,
>> St.Ack
>>
>>
>> Erik Ropez wrote:
>>
>>> Hello,
>>>
>>> I run some tests with C# and Hbase using Thrift. By default
>>> reads/writes were slow (~10/s). After modifying
>>> Thrift.Transport.TSocket class to use BufferedStream, speed increased
>>> a lot (more than 1000 per second). Maybe that helps.
>>>
>>> Thanks,
>>> Erik
>>>
>>>
>>>
>>> On Tue, Dec 9, 2008 at 8:39 PM, Jim Kellerman (POWERSET)
>>> <Jim.Kellerman@microsoft.com> wrote:
>>>
>>>
>>>> As Jonathan Gray said, it is not on our roadmap currently, so
>>>> we have not looked at ways to do it. However, JCC is Apache
>>>> 2.0 license so it is a possible approach.
>>>>
>>>> ---
>>>> Jim Kellerman, Powerset (Live Search, Microsoft Corporation)
>>>>
>>>>
>>>>
>>>>
>>>>> -----Original Message-----
>>>>> From: Wes Chow [mailto:wes.chow@gmail.com]
>>>>> Sent: Tuesday, December 09, 2008 10:26 AM
>>>>> To: hbase-user@hadoop.apache.org
>>>>> Subject: Re: Thrift API vs Java API
>>>>>
>>>>>
>>>>> I stumbled on JCC, the Java->C++ (and Python) bridge that the PyLucene
>>>>> folk are using:
>>>>> http://svn.osafoundation.org/pylucene/trunk/jcc/jcc/README
>>>>>
>>>>> Any thoughts on perhaps using that for the Hadoop family of projects?
>>>>>
>>>>>
>>>>> Wes
>>>>>
>>>>>
>>>>> Jonathan Gray wrote:
>>>>>
>>>>>
>>>>>> I have not used the thrift interface, but know that many others are.
 I
>>>>>> doubt that they are seeing an order of magnitude decrease in
>>>>>>
>>>>>>
>>>>> performance.
>>>>>
>>>>>
>>>>>> What have others seen with thrift?
>>>>>>
>>>>>> There are talks of a new client-side API.  To my knowledge no one
is
>>>>>>
>>>>>>
>>>>> working
>>>>>
>>>>>
>>>>>> on this, so it is not currently scheduled for any upcoming release.
>>>>>>
>>>>>>
>>>>> There
>>>>>
>>>>>
>>>>>> are a number of people who would be interested in it, so please
>>>>>> continue
>>>>>> discussion on the issue if you have an interest in it.
>>>>>>
>>>>>> Here is the open issue regarding this (Pure C/C++ client libraries):
>>>>>> https://issues.apache.org/jira/browse/HBASE-1015
>>>>>>
>>>>>> JG
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: Slava Gorelik [mailto:slava.gorelik@gmail.com]
>>>>>>> Sent: Tuesday, December 09, 2008 6:12 AM
>>>>>>> To: hbase-user@hadoop.apache.org
>>>>>>> Subject: Thrift API vs Java API
>>>>>>>
>>>>>>> Hi.Currently i'm trying to communicate with Hbase from .NET
>>>>>>> environment
>>>>>>> and
>>>>>>> the only option that i have is Thrift.
>>>>>>> I did small comparison (execution time) between Java and Thrift
>>>>>>> (adding
>>>>>>> 1000
>>>>>>> rows with 1k data) and Java is 3.4 ms (average) vs. Thrift that
is 44
>>>>>>> ms
>>>>>>> average.
>>>>>>> It seems that Thrift option is not usable from performance aspect.
>>>>>>>
>>>>>>> Now he question : some one tried to implement Hbase client side
in C++
>>>>>>> /
>>>>>>> .NET ?  Is it feasible ?
>>>>>>>
>>>>>>> Best Regards.
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>



-- 

"Discovering...discovering...we will never cease discovering...
and the end of all our discovering will be
to return to the place where we began
and to know it for the first time."
-T.S. Eliot

Mime
View raw message