hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tsuna <tsuna...@gmail.com>
Subject Re: asynch api mailing list?
Date Mon, 28 Feb 2011 02:13:53 GMT
On Sun, Feb 27, 2011 at 10:29 AM, Hiller, Dean  (Contractor)
<dean.hiller@broadridge.com> wrote:
> I had a question on the hbase asynch api.

Unless other people do not want to discuss asynchbase stuff on this
mailing list, I'm happy to discuss any issue or question here.  The
good thing about using a single list is that it makes it easier to
exchange ideas.

> Maybe I am missing something though????  As a client, I definitely want
> async reads but my writes should only return if there was room in the
> nic buffer to write it out, otherwise it should block and hold up my
> client so my client doesn't have to do any extra coding for a
> PleaseThrottleException.

asynchbase is non-blocking.  It tries hard to never ever block you.
This is very important for high-throughput, low-latency serving
systems.  If you produce writes faster than they can go through the
system, asynchbase will buffer them only up to a certain point
(hardcoded in the code: 10000 edits per region).  Beyond this point,
asynchbase will ask you to throttle yourself since it doesn't wanna
block you to throttle you.

If you have a user-facing server, and this server is accepting a lot
of edits, and because of a problem with this server it's unable to
process edits as fast as it's receiving them, then blocking the server
will cause unpredictable consequences, depending on how you use the
HBase client.  Most likely all your threads will end up blocked and
your server will look like it locked up.

By instead generating the PleaseThrottleException, the asynchbase
client is giving you a chance of becoming aware of the problem and
letting you take action, based on your application's specific needs.
For instance you could take this chance to tell some load-balancing
system sitting in front of your server that you're overwhelmed and
that you would like to receive less traffic.  Or you could close some
persistent connections with some of your clients to force them to
re-open a connection, which will hopefully direct them to another
backend.  Or you could stop accepting new connections until you're no
longer overloaded.

The HBase client has no idea what you do with HBase or how your
application is written, so the most reasonable thing to do IMO is to
throw a PleaseThrottleException and let the application Do The Right
Thing.  Blocking is not an option, especially for a client that
promises to be non-blocking.

Benoit "tsuna" Sigoure
Software Engineer @ www.StumbleUpon.com

View raw message