hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray" <jl...@streamy.com>
Subject RE: HBase and Hadoop MapReduce - Common setups?
Date Tue, 03 Feb 2009 22:42:04 GMT

You're going to want to run your TaskTrackers local to your DataNodes and
RegionServers, again for locality reasons.  That's one of the primary
advantages of MapReduce, moving computation to data.

Otherwise, you are on track.  Of course the setup depends on what you're
doing, but what you describe is on a majority of the HBase setups I'm aware


> -----Original Message-----
> From: Sean Laurent [mailto:organicveggie@gmail.com]
> Sent: Tuesday, February 03, 2009 2:13 PM
> To: hbase-user@hadoop.apache.org
> Subject: HBase and Hadoop MapReduce - Common setups?
> Howdy folks,
> We're evaluating HBase and we're trying to get a good solid picture of
> how
> everything fits together... specifically, we're wondering how people
> commonly setup HBase. I'm imagining you typically run the region
> servers on
> the same machines as the HDFS data nodes to gain data locality
> benefits. And
> from what I've seen on the mailing list, it's typically recommended
> (although it sounds like it's up for debate in terms of SPoF issues) to
> run
> separate machines for the HBaseMaster and NameNode servers.
> Is it something along the following lines?
> 1x HBaseMaster
> 1x HDFS NameNode
> N machines with both HRegionServer and DataNode
> Now what about Hadoop and task trackers? Do people typically run
> completely
> separate clusters for their M/R tasks? Do they run task trackers along
> side
> the region server and data nodes? Or add machines that run TaskTracker
> and
> DataNode servers but ~not~ HRegionServer?
> Any thoughts or opinions would be greatly appreciated!
> -Sean

View raw message