hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: Hbase on EC2 and issues with external access
Date Fri, 20 Mar 2009 14:51:02 GMT
Arber,

See my patch in https://issues.apache.org/jira/browse/HBASE-1279

Now setting hbase.regionserver should work.

J-D


On Fri, Mar 20, 2009 at 8:44 AM, Jean-Daniel Cryans <jdcryans@apache.org> wrote:
> I'm currently looking at the code and I think I see something that may
> be fixing that problem. We have a similar problem here so I'll check
> if it fixes it.
>
> J-D
>
> On Fri, Mar 20, 2009 at 5:51 AM, Yabo-Arber Xu <arber.research@gmail.com> wrote:
>> Hi J-D,
>>
>> I further found that the region server can actually be connected via Web UI
>> from an client outside Amazon network. That further verify the view that
>> Master/Region server both are working, and it's just that Master mapped
>> region servers' external IP into internal IP automatically, and the clients
>> fail to communicate with region server via the mapped internal IP.
>>
>> Do you see any possible solution for this in the near future?
>>
>> Best,
>> Arber
>>
>> On Thu, Mar 19, 2009 at 4:48 PM, Yabo-Arber Xu <arber.research@gmail.com>wrote:
>>
>>> Thx for your explanation. I suspect the reason is that when the master
>>> initiate the contact with region server, region server was recognized
>>> through its internal address ( as both of them are on Amazon network), even
>>> i explicitly put the external address for region server.
>>>
>>> So the consequence is is that the internal address shows in namenode, to
>>> which external clients can not connect.
>>>
>>>
>>> On Thu, Mar 19, 2009 at 4:33 PM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:
>>>
>>>> Getting the "good" host name for a node is kind of a pain. For the
>>>> moment, the implemented solution is that the Master tells the region
>>>> server to override it's known address with what the region server was
>>>> able to contact the master with. It was implemented like this so that
>>>> the region servers stop showing all up as "127.0.0.1" and hadoop was
>>>> doing it the same way.
>>>>
>>>> Now, I know that they changed it on their side, so I'd really really
>>>> like to know with which address the datanodes are showing up in the
>>>> namenode. Internal or external? If it's the external one, I'll dive in
>>>> their code.
>>>>
>>>> Thx,
>>>>
>>>> J-D
>>>>
>>>> On Thu, Mar 19, 2009 at 7:26 PM, Yabo-Arber Xu <arber.research@gmail.com>
>>>> wrote:
>>>> > Hi J-D,
>>>> >
>>>> > Thanks for your reply, and sorry for my late response as I was engaged
>>>> in
>>>> > sth else in the past two days.
>>>> >
>>>> > Yes, I've unlocked the port, and i am actually able to access from the
>>>> web
>>>> > UI with a client not running on EC2 to HBase at example.com:60010. It
>>>> shows
>>>> > all User Tables, but the Region Servers Address is the EC2 internal
>>>> address:
>>>> > domU-12-31-39-00-65-E5.compute-1.internal:60020.
>>>> >
>>>> > I guess the client fails because it can not connect region server, which
>>>> > serves only for an internal IP. However, in hbase-site.xml, I did
>>>> configure
>>>> > with region server explicitly in its external IP.
>>>> >
>>>> >  <property>^M
>>>> >    <name>hbase.regionserver</name>
>>>> >    <value>ec2-67-202-57-127.compute-1.amazonaws.com:60020</value>
>>>> >    <description>The host and port a HBase region server runs
at.^M
>>>> >    </description>
>>>> >  </property>
>>>> >
>>>> > What could I do wrong?
>>>> >
>>>> > Thanks again,
>>>> > Arber
>>>> >
>>>> >
>>>> > On Tue, Mar 17, 2009 at 5:05 AM, Jean-Daniel Cryans <
>>>> jdcryans@apache.org>wrote:
>>>> >
>>>> >> Arber,
>>>> >>
>>>> >> There are security policies with EC2, did you unblock port 60000
for
>>>> >> your own IP address?
>>>> >>
>>>> >> J-D
>>>> >>
>>>> >> On Tue, Mar 17, 2009 at 4:07 AM, Yabo-Arber Xu <
>>>> arber.research@gmail.com>
>>>> >> wrote:
>>>> >> > Hi all,
>>>> >> >
>>>> >> > I set up a small HBase cluster on EC2. It works fine internally
if
>>>> all
>>>> >> the
>>>> >> > applications are within EC2. It, however, does not work if
i am
>>>> trying
>>>> >> run
>>>> >> > shell or client program on a external host. I've been googling
for a
>>>> >> while,
>>>> >> > and found there was similar issues raised before:
>>>> >> >
>>>> >>
>>>> http://www.nabble.com/Hbase-on-EC2-and-issues-with-Amazon-NAT-Internal-Addresses-td21621367.html
>>>> >> ,
>>>> >> >
>>>> >> >
>>>> >> > But it seems no solutions so far. I wonder whether anybody
has made
>>>> >> progress
>>>> >> > on this issue. Any thoughts would be greatly appreciated!
>>>> >> >
>>>> >> >
>>>> >> > Attached with my hbase-site.xml. I am running the master/region
>>>> server
>>>> >> all
>>>> >> > on one instance for testing.
>>>> >> >
>>>> >> > <configuration>
>>>> >> >  <property>
>>>> >> >    <name>hbase.master</name>
>>>> >> >    <value>*domU-12-31-39-00-E0-96.compute-1.internal*:60000</value>
>>>> //It
>>>> >> > does not work even if i changed this the Amazon public IP
>>>> >> >    <description>The host and port that the HBase master
runs at.
>>>> >> >    </description>
>>>> >> >  </property>
>>>> >> >
>>>> >> >  <property>
>>>> >> >    <name>hbase.rootdir</name>
>>>> >> >
>>>> >> >
>>>> >>
>>>> <value>hdfs://domU-12-31-39-00-E0-96.compute-1.internal:54310/hbase</value>
>>>> >> >    <description>The directory shared by region servers.
>>>> >> >    </description>
>>>> >> >  </property>
>>>> >> > </configuration>
>>>> >> >
>>>> >> >
>>>> >> > Thanks for your attention,
>>>> >> > Arber
>>>> >> >
>>>> >>
>>>> >
>>>>
>>>
>>
>

Mime
View raw message