hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Graham <billgra...@gmail.com>
Subject Re: Scheduling map/reduce jobs
Date Wed, 05 Jan 2011 17:40:07 GMT
Take a look at Oozie or Azkaban:


On Wed, Jan 5, 2011 at 9:35 AM, Peter Veentjer <alarmnummer@gmail.com> wrote:
> He Guys,
> although it isn't completely related to HBase. Is there support for
> scheduling map reduce jobs?
> E.g. I want to do a map reduce job that automatically removes certain
> elements from hbase and perhaps some additional cleanup (I know that there
> is support for lease times).
> I could have every node in the cluster schedule this job every hour, but if
> there are n nodes I don't want this job to be running n times. Only 1 time
> would be sufficient. I can create some checking logic for scheduling a map
> reduce job where a job that should not be run is rejected/ignored, but if
> there is something out of the box.
> Another question:
> We are building an environment where client provided plugins can be deployed
> and obey some kind of SLA (e.g. we want it to be running on 2 nodes at
> least). So the old plugin needs to be undeployed, the jars of the new plugin
> need to be copied on demand to the nodes and the new plugin needs to be
> deployed. Is there functionality for this behavior out of the box or does it
> need to be implemented by ourselves?

View raw message