hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Segel <michael_se...@hotmail.com>
Subject RE: hbase with multiple interfaces
Date Thu, 29 Apr 2010 11:39:38 GMT

Is sns12-virt the actual machine name? 
What do you get if you type in uname -a?

The problem with Hadoop and of course HBase is that they determine their own IP network based
on the machine's actual name, so that even if you have multiple interfaces, the nodes will
choose the interface that matches the machine name. (IMHO this is a defect that should be
fixed.) 

The use case is that you may want to expose a 1GBe nic to the outside world for access to
the cloud and then use a 10GBe interface for inter-cloud communication.

Just IMHO.

-Mike

> Date: Wed, 28 Apr 2010 20:11:37 -0400
> From: dgarbuzov@gmail.com
> To: hbase-user@hadoop.apache.org
> CC: jterrace@cs.princeton.edu
> Subject: Re: hbase with multiple interfaces
> 
> I'm getting the same results without hbase.regionserver.dns.interface 
> set and with it explicitly set  to "default"
> 
> hbase-site.xml:
> <configuration>
>     <property>
>         <name>hbase.rootdir</name>
>         <value>hdfs://sns12.cs.princeton.edu:8020/hbase</value>
>     </property>
>     <property>
>         <name>hbase.master</name>
>         <value>sns12-virt.CS.Princeton.EDU:60000</value>
>     </property>
>     <property>
>         <name>hbase.cluster.distributed</name>
>         <value>true</value>
>     </property>
>     <property>
>         <name>hbase.zookeeper.quorum</name>
>         
> <value>sns12-virt.CS.Princeton.EDU,sns13-virt.CS.Princeton.EDU,sns14-virt.CS.Princeton.EDU</value>
>     </property>
> </configuration>
> 
> 
> Ryan Rawson wrote:
> > If you are using the "hbase.regionserver.dns.interface" option this is
> > what would happen - binding to a specific interface.
> >
> > What is your config looking like?
> >
> > On Wed, Apr 28, 2010 at 4:54 PM, dmitri garbuzov <dgarbuzov@gmail.com> wrote:
> >   
> >> Doesn't look like it's binding to 0.0.0.0
> >>
> >> sudo lsof -i :60020
> >> COMMAND   PID   USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
> >> java    22530 hadoop   37u  IPv4 81808593      0t0  TCP
> >> sns12-virt.CS.Princeton.EDU:60020 (LISTEN)
> >>
> >> sudo netstat -an | grep 60020
> >> tcp        0      0 128.112.7.112:60020         0.0.0.0:*
> >> LISTEN
> >>
> >>
> >> Ryan Rawson wrote:
> >>     
> >>> HBase does bind to 0.0.0.0:60020.  It uses the hostname to report this
> >>> to META and that is how other people find the region servers.
> >>>
> >>> Are you not seeing this?
> >>>
> >>> On Wed, Apr 28, 2010 at 4:15 PM, dmitri garbuzov <dgarbuzov@gmail.com>
> >>> wrote:
> >>>
> >>>       
> >>>> Hi all,
> >>>>
> >>>> I'm trying to set up hbase on a cluster where each machine has multiple
> >>>> interfaces.  Hdfs worked right away since I could just set the name
node
> >>>> to
> >>>> listen on 0.0.0.0, but hbase region servers seem to pick one of the
> >>>> interfaces at random, ignoring the hbase.regionserver.dns.interface
> >>>> option.
> >>>>  Is the dns.interface setting supposed to make hbase bind to a specific
> >>>> interface? Is there anything special I need to do to get this to work?
> >>>>
> >>>> dmitri
> >>>>
> >>>>
> >>>>         
> >>>       
> >>     
> >
> >   
> 
 		 	   		  
_________________________________________________________________
Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox.
http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_1
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message