hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: REST/Stargate Performance
Date Tue, 20 Apr 2010 21:54:02 GMT
This is the way it is with REST. You have HTTP transaction overheads for each access to a (path
specified) resource. Multiple clients and Stargate instances will help. REST/WS is best suited
for the case where you will have thousands of concurrent clients making fairly infrequent
requests along the order of what you mention below, not a few clients making a large number
of requests. 

For the latter case, using a streaming interface like Thrift, or you can consider using (or
helping to build) the new Avro based connector described in HBASE-2400. 

Also, I can get several hundred reads per second using scanners and batching, and I can do
several hundred puts per second using Stargate's multi-put capability. Neither scanners nor
multi-put are all that REST-ful.

I recently profiled Stargate with jprofiler. The Jersey JAX-RS framework adds a bit of overhead,
URL path parsing and the like, but it was on the order of a few percent only. The bulk of
the time is spent handling client I/O or I/O to/from the region servers, which is as it should

Search the hbase-users@ mailing list for more detail on which connector to use for what architecture
and use case. 

   - Andy

> From: Pat Mao <drkul@hotmail.com>
> Subject: REST/Stargate Performance
> To: hbase-user@hadoop.apache.org
> Date: Tuesday, April 20, 2010, 2:42 PM
> Hi,
> I'm having performance issues with the Stargate RESTful web
> service.  I need to use multiple languages to
> communicate with HBase and prefer to use REST over
> Thrift.  But when using Stargate, I'm getting somewhere
> along the lines of 20-50 reads per second (single threaded
> test), which is MUCH worse than using the Java API (~2k) or
> even Thrift (~1k).  I know that Stargate will be the
> slowest, but I was wondering if this is the type of
> performance I should be expecting or if I'm doing something
> wrong.  We even tried writing our own Java web service
> and got similarly poor results.  Also, we're using
> HBase 0.20.3 with compressed tables.
> Any help would be greatly appreciated.
> Thanks in advance,
> Pat
> _________________________________________________________________
> The New Busy is not the old busy. Search, chat and e-mail
> from your inbox.
> http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_3


View raw message