hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marc Harris <mhar...@jumptap.com>
Subject Multiple NICs on hbase nodes - regionservers register with master using wrong address
Date Tue, 04 Mar 2008 14:46:11 GMT
I am trying to set up an hbase cluster, and the nodes are unable to
communicate with each other. I have the following slightly unusual
setup:

- Each machine has two NICs:

NIC1:
- accessible outside the network
- lower bandwidth
- restricted ports
- listed in DNS

NIC2:
- accessible only inside the network
- higher bandwidth
- all ports open
- listed in /etc/hosts, not DNS (for some reason; I have no control over
this).

What I am finding is that even if I use the servers internal names in
the property dfs.datanode.dns.interface, the region servers announce
themselves to the master using the external NIC address. Even if I use
the numeric IP address of the internal NIC, it still uses the internal
IP address.

Tracing this through with a debugger, it looks like
HRegionServer.getThisIP() which calls org.apache.hadoop.net.DNS does not
handle the situation correctly. Perhaps if the external NIC was in DNS
it would work; I don't know. 

So a 2 part question:
1) Why does the code do any of this? Why not just assume that if the
user has supplied a value for dfs.datanode.dns.interface, then it is
correct, and the value to use?
2) Is there any way I can work around this problem?

Somehow datanode manages to avoid this problem. It registers to the
namenode using the correct (internal) NIC.

Thanks for any help you can provide.


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message