whirr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Segel <michael_se...@hotmail.com>
Subject Trying to build out Storm / CDH Yarn
Date Fri, 02 Aug 2013 19:19:56 GMT

I'm still fairly new to Whirr and I am starting to run in to some issues that  I would love
to get some suggestions to see if want I want is feasible…

I'm trying to use Whirr to build out a cluster on AWS using Storm , CDH 4 running YARN , HBase
and Yahoo! 's storm-yarn code.

So far, I've cobbled together a recipe to build the CDH4 Yarn cluster w HBase and ZK running.

My next task is to add my own resource file to put the storm-yarn jar on every node. 

I think I have a handle on that… 

But now I am hitting a wall… 

Need advice on how to load Storm on the cluster.
I need to create a single node as a Nimbus node, not to mention their own ZK cluster (1 or
3 nodes)

Here's the rub. Installing HBase, I run a resource script to install the ZK for HBase.
If I want to keep the storm and HBase separate is there a way to identify different ZK quorums?

Is there a way to integrate Whirr so that it could launch a Storm script?
(Do this first in my storm-yarn resource script?)
If this works, how can I get the cluster to see the other node?

Is there any documentation on how to write a service ? 


View raw message