hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: Hbase on EC2 and issues with external access
Date Fri, 20 Mar 2009 12:44:31 GMT
I'm currently looking at the code and I think I see something that may
be fixing that problem. We have a similar problem here so I'll check
if it fixes it.

J-D

On Fri, Mar 20, 2009 at 5:51 AM, Yabo-Arber Xu <arber.research@gmail.com> wrote:
> Hi J-D,
>
> I further found that the region server can actually be connected via Web UI
> from an client outside Amazon network. That further verify the view that
> Master/Region server both are working, and it's just that Master mapped
> region servers' external IP into internal IP automatically, and the clients
> fail to communicate with region server via the mapped internal IP.
>
> Do you see any possible solution for this in the near future?
>
> Best,
> Arber
>
> On Thu, Mar 19, 2009 at 4:48 PM, Yabo-Arber Xu <arber.research@gmail.com>wrote:
>
>> Thx for your explanation. I suspect the reason is that when the master
>> initiate the contact with region server, region server was recognized
>> through its internal address ( as both of them are on Amazon network), even
>> i explicitly put the external address for region server.
>>
>> So the consequence is is that the internal address shows in namenode, to
>> which external clients can not connect.
>>
>>
>> On Thu, Mar 19, 2009 at 4:33 PM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:
>>
>>> Getting the "good" host name for a node is kind of a pain. For the
>>> moment, the implemented solution is that the Master tells the region
>>> server to override it's known address with what the region server was
>>> able to contact the master with. It was implemented like this so that
>>> the region servers stop showing all up as "127.0.0.1" and hadoop was
>>> doing it the same way.
>>>
>>> Now, I know that they changed it on their side, so I'd really really
>>> like to know with which address the datanodes are showing up in the
>>> namenode. Internal or external? If it's the external one, I'll dive in
>>> their code.
>>>
>>> Thx,
>>>
>>> J-D
>>>
>>> On Thu, Mar 19, 2009 at 7:26 PM, Yabo-Arber Xu <arber.research@gmail.com>
>>> wrote:
>>> > Hi J-D,
>>> >
>>> > Thanks for your reply, and sorry for my late response as I was engaged
>>> in
>>> > sth else in the past two days.
>>> >
>>> > Yes, I've unlocked the port, and i am actually able to access from the
>>> web
>>> > UI with a client not running on EC2 to HBase at example.com:60010. It
>>> shows
>>> > all User Tables, but the Region Servers Address is the EC2 internal
>>> address:
>>> > domU-12-31-39-00-65-E5.compute-1.internal:60020.
>>> >
>>> > I guess the client fails because it can not connect region server, which
>>> > serves only for an internal IP. However, in hbase-site.xml, I did
>>> configure
>>> > with region server explicitly in its external IP.
>>> >
>>> >  <property>^M
>>> >    <name>hbase.regionserver</name>
>>> >    <value>ec2-67-202-57-127.compute-1.amazonaws.com:60020</value>
>>> >    <description>The host and port a HBase region server runs at.^M
>>> >    </description>
>>> >  </property>
>>> >
>>> > What could I do wrong?
>>> >
>>> > Thanks again,
>>> > Arber
>>> >
>>> >
>>> > On Tue, Mar 17, 2009 at 5:05 AM, Jean-Daniel Cryans <
>>> jdcryans@apache.org>wrote:
>>> >
>>> >> Arber,
>>> >>
>>> >> There are security policies with EC2, did you unblock port 60000 for
>>> >> your own IP address?
>>> >>
>>> >> J-D
>>> >>
>>> >> On Tue, Mar 17, 2009 at 4:07 AM, Yabo-Arber Xu <
>>> arber.research@gmail.com>
>>> >> wrote:
>>> >> > Hi all,
>>> >> >
>>> >> > I set up a small HBase cluster on EC2. It works fine internally
if
>>> all
>>> >> the
>>> >> > applications are within EC2. It, however, does not work if i am
>>> trying
>>> >> run
>>> >> > shell or client program on a external host. I've been googling
for a
>>> >> while,
>>> >> > and found there was similar issues raised before:
>>> >> >
>>> >>
>>> http://www.nabble.com/Hbase-on-EC2-and-issues-with-Amazon-NAT-Internal-Addresses-td21621367.html
>>> >> ,
>>> >> >
>>> >> >
>>> >> > But it seems no solutions so far. I wonder whether anybody has
made
>>> >> progress
>>> >> > on this issue. Any thoughts would be greatly appreciated!
>>> >> >
>>> >> >
>>> >> > Attached with my hbase-site.xml. I am running the master/region
>>> server
>>> >> all
>>> >> > on one instance for testing.
>>> >> >
>>> >> > <configuration>
>>> >> >  <property>
>>> >> >    <name>hbase.master</name>
>>> >> >    <value>*domU-12-31-39-00-E0-96.compute-1.internal*:60000</value>
>>> //It
>>> >> > does not work even if i changed this the Amazon public IP
>>> >> >    <description>The host and port that the HBase master
runs at.
>>> >> >    </description>
>>> >> >  </property>
>>> >> >
>>> >> >  <property>
>>> >> >    <name>hbase.rootdir</name>
>>> >> >
>>> >> >
>>> >>
>>> <value>hdfs://domU-12-31-39-00-E0-96.compute-1.internal:54310/hbase</value>
>>> >> >    <description>The directory shared by region servers.
>>> >> >    </description>
>>> >> >  </property>
>>> >> > </configuration>
>>> >> >
>>> >> >
>>> >> > Thanks for your attention,
>>> >> > Arber
>>> >> >
>>> >>
>>> >
>>>
>>
>

Mime
View raw message