hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiller, Dean x66079" <dean.hil...@broadridge.com>
Subject RE: On startup is there a hook to start another framework?
Date Wed, 13 Jul 2011 13:25:34 GMT
Hmmmm, how about coprocessors, will the coprocessor always fire on the primary node?

If so, then I am thinking I would just keep writing to the account object itself and the account
on it's node would then do all the processing(though I would want this done on that same thread
so any error is reported back to the original put)

But then I am not sure I can put/get from other tables in the coprocessor, can I?  I don't
need to write to the account at all...just trying to get a trigger to go off and use the data
passed through the trigger.


Our main processing is we have 10,000,000 activities come in and each one we want to just
pass a DTO to the primary node with that account which is then responsible for pairing that
activity.  We have millions of accounts and about 126 activities per account so we would like
to have mini-databases so to speak in that each account has it's own indexes for activities,
and a few other tables(around 6) and all these get modified when an activity comes into the
account.  Any ideas appreciated.


-----Original Message-----
From: saint.ack@gmail.com [mailto:saint.ack@gmail.com] On Behalf Of Stack
Sent: Monday, July 11, 2011 1:16 PM
To: user@hbase.apache.org
Subject: Re: On startup is there a hook to start another framework?

On Mon, Jul 11, 2011 at 8:16 AM, Hiller, Dean  x66079
<dean.hiller@broadridge.com> wrote:
> 1.       How can I startup a framework in hbase?

Use coprocessors or, subclass it (the latter is not recommended -- it
becomes a pain fast).

> 2.       How can I locate the primary node the data is on and the secondary nodes?

For a file?  At HBase level, we don't really have a locality notion.
You'd have to go to HDFS to find the blocks that comprise a file and
where the replicas are located.

> 3.       Also, I would like all function calls for the same account to occur on the
primary node or at least the same node if I have a replication factor of 3 so that I can synchronize
on activities coming in for the same account

I'm not sure how to answer this one.

This message and any attachments are intended only for the use of the addressee and may contain
information that is privileged and confidential. If the reader of the message is not the intended
recipient or an authorized representative of the intended recipient, you are hereby notified
that any dissemination of this communication is strictly prohibited. If you have received
this communication in error, please notify us immediately by e-mail and delete the message
and any attachments from your system.

View raw message