hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yabo-Arber Xu <arber.resea...@gmail.com>
Subject Re: Hbase on EC2 and issues with external access
Date Fri, 20 Mar 2009 19:10:57 GMT
Hi J-D,

Thank you very much. I shall try it. I am still using hbase-0.18.1. Do I
have upgrade it to 0.19.x and then apply the patch?

Best,
Arber

On Fri, Mar 20, 2009 at 7:51 AM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:

> Arber,
>
> See my patch in https://issues.apache.org/jira/browse/HBASE-1279
>
> Now setting hbase.regionserver should work.
>
> J-D
>
>
> On Fri, Mar 20, 2009 at 8:44 AM, Jean-Daniel Cryans <jdcryans@apache.org>
> wrote:
> > I'm currently looking at the code and I think I see something that may
> > be fixing that problem. We have a similar problem here so I'll check
> > if it fixes it.
> >
> > J-D
> >
> > On Fri, Mar 20, 2009 at 5:51 AM, Yabo-Arber Xu <arber.research@gmail.com>
> wrote:
> >> Hi J-D,
> >>
> >> I further found that the region server can actually be connected via Web
> UI
> >> from an client outside Amazon network. That further verify the view that
> >> Master/Region server both are working, and it's just that Master mapped
> >> region servers' external IP into internal IP automatically, and the
> clients
> >> fail to communicate with region server via the mapped internal IP.
> >>
> >> Do you see any possible solution for this in the near future?
> >>
> >> Best,
> >> Arber
> >>
> >> On Thu, Mar 19, 2009 at 4:48 PM, Yabo-Arber Xu <
> arber.research@gmail.com>wrote:
> >>
> >>> Thx for your explanation. I suspect the reason is that when the master
> >>> initiate the contact with region server, region server was recognized
> >>> through its internal address ( as both of them are on Amazon network),
> even
> >>> i explicitly put the external address for region server.
> >>>
> >>> So the consequence is is that the internal address shows in namenode,
> to
> >>> which external clients can not connect.
> >>>
> >>>
> >>> On Thu, Mar 19, 2009 at 4:33 PM, Jean-Daniel Cryans <
> jdcryans@apache.org>wrote:
> >>>
> >>>> Getting the "good" host name for a node is kind of a pain. For the
> >>>> moment, the implemented solution is that the Master tells the region
> >>>> server to override it's known address with what the region server was
> >>>> able to contact the master with. It was implemented like this so that
> >>>> the region servers stop showing all up as "127.0.0.1" and hadoop was
> >>>> doing it the same way.
> >>>>
> >>>> Now, I know that they changed it on their side, so I'd really really
> >>>> like to know with which address the datanodes are showing up in the
> >>>> namenode. Internal or external? If it's the external one, I'll dive
in
> >>>> their code.
> >>>>
> >>>> Thx,
> >>>>
> >>>> J-D
> >>>>
> >>>> On Thu, Mar 19, 2009 at 7:26 PM, Yabo-Arber Xu <
> arber.research@gmail.com>
> >>>> wrote:
> >>>> > Hi J-D,
> >>>> >
> >>>> > Thanks for your reply, and sorry for my late response as I was
> engaged
> >>>> in
> >>>> > sth else in the past two days.
> >>>> >
> >>>> > Yes, I've unlocked the port, and i am actually able to access from
> the
> >>>> web
> >>>> > UI with a client not running on EC2 to HBase at example.com:60010.
> It
> >>>> shows
> >>>> > all User Tables, but the Region Servers Address is the EC2 internal
> >>>> address:
> >>>> > domU-12-31-39-00-65-E5.compute-1.internal:60020.
> >>>> >
> >>>> > I guess the client fails because it can not connect region server,
> which
> >>>> > serves only for an internal IP. However, in hbase-site.xml, I did
> >>>> configure
> >>>> > with region server explicitly in its external IP.
> >>>> >
> >>>> >  <property>^M
> >>>> >    <name>hbase.regionserver</name>
> >>>> >    <value>ec2-67-202-57-127.compute-1.amazonaws.com:60020</value>
> >>>> >    <description>The host and port a HBase region server runs
at.^M
> >>>> >    </description>
> >>>> >  </property>
> >>>> >
> >>>> > What could I do wrong?
> >>>> >
> >>>> > Thanks again,
> >>>> > Arber
> >>>> >
> >>>> >
> >>>> > On Tue, Mar 17, 2009 at 5:05 AM, Jean-Daniel Cryans <
> >>>> jdcryans@apache.org>wrote:
> >>>> >
> >>>> >> Arber,
> >>>> >>
> >>>> >> There are security policies with EC2, did you unblock port
60000
> for
> >>>> >> your own IP address?
> >>>> >>
> >>>> >> J-D
> >>>> >>
> >>>> >> On Tue, Mar 17, 2009 at 4:07 AM, Yabo-Arber Xu <
> >>>> arber.research@gmail.com>
> >>>> >> wrote:
> >>>> >> > Hi all,
> >>>> >> >
> >>>> >> > I set up a small HBase cluster on EC2. It works fine internally
> if
> >>>> all
> >>>> >> the
> >>>> >> > applications are within EC2. It, however, does not work
if i am
> >>>> trying
> >>>> >> run
> >>>> >> > shell or client program on a external host. I've been
googling
> for a
> >>>> >> while,
> >>>> >> > and found there was similar issues raised before:
> >>>> >> >
> >>>> >>
> >>>>
> http://www.nabble.com/Hbase-on-EC2-and-issues-with-Amazon-NAT-Internal-Addresses-td21621367.html
> >>>> >> ,
> >>>> >> >
> >>>> >> >
> >>>> >> > But it seems no solutions so far. I wonder whether anybody
has
> made
> >>>> >> progress
> >>>> >> > on this issue. Any thoughts would be greatly appreciated!
> >>>> >> >
> >>>> >> >
> >>>> >> > Attached with my hbase-site.xml. I am running the master/region
> >>>> server
> >>>> >> all
> >>>> >> > on one instance for testing.
> >>>> >> >
> >>>> >> > <configuration>
> >>>> >> >  <property>
> >>>> >> >    <name>hbase.master</name>
> >>>> >> >
>  <value>*domU-12-31-39-00-E0-96.compute-1.internal*:60000</value>
> >>>> //It
> >>>> >> > does not work even if i changed this the Amazon public
IP
> >>>> >> >    <description>The host and port that the HBase
master runs at.
> >>>> >> >    </description>
> >>>> >> >  </property>
> >>>> >> >
> >>>> >> >  <property>
> >>>> >> >    <name>hbase.rootdir</name>
> >>>> >> >
> >>>> >> >
> >>>> >>
> >>>>
> <value>hdfs://domU-12-31-39-00-E0-96.compute-1.internal:54310/hbase</value>
> >>>> >> >    <description>The directory shared by region servers.
> >>>> >> >    </description>
> >>>> >> >  </property>
> >>>> >> > </configuration>
> >>>> >> >
> >>>> >> >
> >>>> >> > Thanks for your attention,
> >>>> >> > Arber
> >>>> >> >
> >>>> >>
> >>>> >
> >>>>
> >>>
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message