whirr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: HBase on EC2: Could not resolve the DNS name of ...
Date Thu, 13 Oct 2011 03:02:43 GMT
This is an EC2 limitation. "External" IP addresses are a fiction of DNS and NAT. When you query
the OS for the IP address of the local interfaces, you get internal IP addresses (10.x.x.x),
and reverse resolution of same gives internal DNS names. This is all the information we have
to set up cluster bookeeping in Zookeeper, and all we have to provide to clients. 

There is an instance metadata service that EC2 offers, where one can fetch instance metadata
via HTTP using a well known IP address in the EC2 infrastructure, but this is a site-specific
hack that HBase can't avail itself of. We use the Hadoop net API for getting hostname identity.

The only effective solutions are variations of what Karel describes below. You can add entries
to /etc/hosts on the clients that map internal host names to the public IP addresses that
one gets back via ec2-describe-instances, or set up a local DNS server and dynamically update
it accordingly.

     - Andy



>________________________________
>From: Karel Vervaeke <karel@outerthought.org>
>To: user@whirr.apache.org
>Sent: Wednesday, October 5, 2011 3:04 AM
>Subject: Re: HBase on EC2: Could not resolve the DNS name of ...
>
>
>I believe a workaround is to add entries to /etc/hosts, mapping the internal hostname
(domU-.....internal) to the public ip address of the nodes. Not really sure though...
>
>
>On Tue, Oct 4, 2011 at 3:49 PM, Andrei Savu <savu.andrei@gmail.com> wrote:
>
>This is a known issue and we don't have a workaround. 
>>
>>
>>Would it be possible to use the rest interface from the local machine? How are you
planning to use HBase?
>>
>>
>>As a side note you can start in the same security group an "empty" machine using the
noop role. 
>>
>>-- Andrei Savu / andreisavu.ro
>>
>>
>>On Tue, Oct 4, 2011 at 4:43 PM, Jim R. Wilson <wilson.jim.r@gmail.com> wrote:
>>
>>Hi all,
>>>
>>>
>>>First, let me thank you for adding 0.90.3 support to Whirr 0.6.0 - this is a big
help!
>>>
>>>
>>>I'm trying to run a small demo cluster based on the 0.90.3-ec2 recipe.  The cluster
seems to be working, because when I SSH into a node and use the hbase shell (/usr/local/hbase-0.90.3/bin/hbase
shell), commands seem to work as expected.
>>>
>>>
>>>However, when I try to connect from my local host by using the generated hbase-site.xml
over the proxy tunnel created by hbase-proxy.sh, I get this error message:
>>>
>>>
>>>hbase(main):001:0> status
>>>11/10/04 09:37:12 ERROR hbase.HServerAddress: Could not resolve the DNS name of
domU-<...>.compute-1.internal:60000
>>>
>>>
>>>ERROR: java.lang.IllegalArgumentException: Could not resolve the DNS name of domU-<...>.compute-1.internal:60000
>>>
>>>
>>>Am I doing something wrong?  Is this a known issue?  Thanks,
>>>
>>>
>>>-- Jim R Wilson (jimbojw)
>>
>
>
>
>-- 
>Karel Vervaeke
>http://outerthought.org/
>Open Source Content Applications
>Makers of Kauri, Daisy CMS and Lily
>
>
>
Mime
View raw message