hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: EC2 scripts
Date Thu, 19 Nov 2009 18:07:53 GMT
I just performed the following steps:

1) Install pristine RHEL 5 (well, CentOS) system in VM.

2) Download and unzip the EC2 AMI and API tools zipfiles.

3) Put the tools on the path and set EC2_HOME in the environment.

4) Pristine checkout of HBase trunk.

5) Configure hbase-ec2-env.sh 

6) ./bin/hbase-ec2 launch-cluster testcluster 3 3

7) ./bin/hbase-ec2 login testcluster

8) hbase shell

9) status 'simple'

3 live servers
    domU-12-31-39-09-75-11.compute-1.internal:60020 1258653694915
        requests=0, regions=1, usedHeap=29, maxHeap=987
    domU-12-31-39-01-AC-31.compute-1.internal:60020 1258653709041
        requests=0, regions=1, usedHeap=29, maxHeap=987
    domU-12-31-39-01-B0-91.compute-1.internal:60020 1258653706411
        requests=0, regions=0, usedHeap=27, maxHeap=987
0 dead servers

I would be very interested to see what is in the HBase logs on your slaves, and also the full
output of the cluster launch if you can capture that into a buffer and save it to a file.


    - Andy

From: Andrew Purtell <apurtell@apache.org>
To: hbase-user@hadoop.apache.org
Sent: Thu, November 19, 2009 9:28:51 AM
Subject: Re: EC2  scripts

So it seems the slaves are not checking in with the master for some reason. 

Can you provide the log content from the slave instances in /mnt/hbase/logs/*.log ? 

What is the output of 'hadoop dfsadmin -report'? 

Thanks for trying this out Naresh. 

   - Andy

From: Naresh Rapolu <nrapolu@purdue.edu>
To: hbase-user@hadoop.apache.org
Sent: Thu, November 19, 2009 9:09:54 AM
Subject: Re: EC2  scripts

Thanks Andrew,

We`ve made a private instance with our keys,  by altering your public hbase AMI and used it
to launch the cluster instances.
We are now able to do  "ssh slave" from master.  But, still the problem persists.  From the
hbase shell of master,  "status 'simple' " returns  0 live servers.
Also, while creating a new table from the hbase shell, the following  error pops up.

hbase(main):001:0> create 'dummy', 'dummyfamily'
NativeException: org.apache.hadoop.hbase.master.NotAllMetaRegionsOnlineException: org.apache.hadoop.hbase.master.NotAllMetaRegionsOnlineException
       at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:750)
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
       at java.lang.reflect.Method.invoke(Method.java:597)
       at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:648)
       at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)

       from sun/reflect/NativeConstructorAccessorImpl.java:-2:in `newInstance0'
       from sun/reflect/NativeConstructorAccessorImpl.java:39:in `newInstance'
       from sun/reflect/DelegatingConstructorAccessorImpl.java:27:in `newInstance'
       from java/lang/reflect/Constructor.java:513:in `newInstance'
       from org/apache/hadoop/hbase/RemoteExceptionHandler.java:94:in `decodeRemoteException'
       from org/apache/hadoop/hbase/client/HBaseAdmin.java:208:in `createTableAsync'
       from org/apache/hadoop/hbase/client/HBaseAdmin.java:165:in `createTable'
      -- We`ve checked and ensured that all slaves are online with  HRegionServer  daemons.

Btw,  Hadoop is working well,  we are able to put in large files into the dfs.   Problem seems
to be with Hbase alone.

Let me know if you need any more log messages.

Naresh Rapolu.

Andrew Purtell wrote:
> 1) How long did you wait before logging on to the master and checking status? I bet if
you checked the slave instances status at that time (e.g. using elasticfox) they were still
pending startup. The slaves are started last and EC2 may not start them right away nor all
at once.
> 2) The way private keys work with EC2 instances you must supply the EC2 private key using
-i unless you also use this as your default private key, set up the appropriate config (man
5 ssh_config), or build your own private instance with your default key already installed.

> On Wed Nov 18th, 2009 9:49 PM PST Naresh Rapolu wrote:
>> Hello ,
>> We were trying to use the  src/contrib/ec2  scripts.  All the instances started well,
 but from  hbase shell, on the master ,  a   " status 'simple' "   listed  0  live servers.
Also , we are not able to ssh into any of the slave instances from the master,  without using
 " shh -i  private_key.pem  ".   (ie)  ssh is not configured to be used without a password
and with automatic keys.   I guess, the instances still maintain the creator`s private key.
  Did they work for any others, apart from the creator ?   Even the  JIRA page shows that
 it didnt completely work  for  St.Ack.
>> Thanks,
>> Naresh Rapolu

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message