hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Harvey <danharve...@gmail.com>
Subject Re: async table updates?
Date Wed, 23 Feb 2011 23:37:13 GMT
You could use something like the row log library from the Lilly project to queue the processing
from one table to the other every time you do a put in HBase. As the queue is in HBase itself
too it's all atomic, I'm sure you could get it working with minimal lag between the two tables.


On 23 Feb 2011, at 22:40, Ryan Rawson <ryanobjc@gmail.com> wrote:

> In thrift there is a 'oneway' or 'async' or 'fire and forget' call
> type.  I cant recommend those kinds of approaches, since once your
> system runs into problems you have no feedback.  So if you are asking
> for a one shot, no reply "assume it worked" call, we don't have one
> (nor would I wish that hell on anyone).
> So it boils down to your application architecture.  I'm sure you can
> figure out how to get notifications via callbacks (with background
> threads or threadpools).
> Or did you mean something else?
> -ryan
> On Wed, Feb 23, 2011 at 2:31 PM, Vishal Kapoor
> <vishal.kapoor.in@gmail.com> wrote:
>> I have two tables called LIVE and MASTER.
>> LIVE reports about the MASTER activity and I need to process records in LIVE
>> almost real time( some business logic)
>> if I need to store the activity of entities reported by LIVE rows in MASTER
>> I could process my data in LIVE and at the same time update the MASTER's
>> "LIVE REPORTED" attribute with value of current timestamp.
>> but I was wondering if I can update the MASTER table in a async call so that
>> I don't compromise on the speed.
>> was wondering if anyone tried to achieve this or even if it is advisable...
>> thanks,
>> Vishal Kapoor

View raw message