hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stack <st...@duboce.net>
Subject Re: Is TaskTracker required for hbase?
Date Fri, 11 Dec 2009 18:15:52 GMT
Hey Michael:

Combining regionserver and datanode daemons is an interesting idea.  An
initial cut shouldn't be too hard.  You'd have the two daemons communicate
via RPC as they do now.  For an example, look at hbase tests where we spin
up minidfscluster and a minihbasecluster all in the one JVM.

Integrating ZooKeeper makes less sense.  It still has System.exit calls
embedded.  Also, no matter how you slice it, its in the nature of the
zookeeper cluster to be orthogonal to the running of the hbase+datanode
cluster.

I am with Edward, both that its a cool idea, but also that this is not
something to be rushed.  I'd suggest that the  effort conflating processes
would be better spent on study of the running system understanding how they
interact in your environment, developing a necessary balance between all the
running parts.  We can help you out if you go this path.  If you make a
frankenstein of hbase + datanodes, while it'd be cute, we'll be less able to
assist especially when time is short.

Good luck Michael,
St.Ack




On Fri, Dec 11, 2009 at 6:47 AM, Edward Capriolo <edlinuxguru@gmail.com>wrote:

> If you do not have much time to experiment, you prob do not have time
> for this, because you are going to have to build your own code to run
> all the components in one jvm. If you are unfamiliar with the
> codebases this takes getting used to, also running the ant test
> targets take some time. so I can't immagine such a drastic recoding
> would save you time.
>
> On 12/11/09, Michał Podsiadłowski <podsiadlowski@gmail.com> wrote:
> > 2009/12/11 Ryan Rawson <ryanobjc@gmail.com>
> >
> >> They are not necessary.
> >>
> >> HBase only depends on HDFS for storage, so you will need datanode,
> >> namenode and the hbase processes.
> >>
> >>
> >>
> >>
> > Hi Ryan
> > Thanks for answer. This will make our lives easier.
> > I've got another question which I've posted yesterday about combining
> > datanode and hbase processes (including zookeeper) into single JVM
> process.
> > Would you be so kind and drop me a few words about this since your our
> hbase
> > guru. I really need to know how this can behave - we don't have much time
> > for experiments and any clues can occure indispensable for us.
> >
> > Thanks,
> > Michael
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message