hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fei Dong <dongfei...@gmail.com>
Subject Re: NoClassDefFoundError when running Hadoop with HBase
Date Tue, 24 Jan 2012 21:22:10 GMT
Hi Stack,

I try ./zkCli.sh -server 10.114.45.186:2181, it works. I will list
hbase-site.xml, zookeeper log, and Hadoop logs. Could you take a look?
Thanks.

 hbase-site.xml:
<property>
    <name>hbase.zookeeper.property.clientPort</name>
    <value>2181</value>
  </property>
  <property>
    <name>hbase.zookeeper.quorum</name>
    <value>10.114.45.186</value>
  </property>
  <property>
    <name>hbase.zookeeper.property.dataDir</name>
    <value>/mnt/zookeeper</value>
  </property>
  <property>
    <name>hbase.zookeeper.property.maxClientCnxns</name>
    <value>1000</value>
  </property>

Zookeeper log shows:
2012-01-24 16:17:35,665 INFO
org.apache.zookeeper.server.NIOServerCnxn: Accepted socket connection
from /127.0.0.1:41807
2012-01-24 16:17:35,668 INFO
org.apache.zookeeper.server.NIOServerCnxn: Client attempting to
establish new session at /127.0.0.1:41807
2012-01-24 16:17:35,670 INFO
org.apache.zookeeper.server.NIOServerCnxn: Established session
0x13510f94d2d002c with negotiated timeout 180000 for client
/127.0.0.1:41807
2012-01-24 16:19:12,313 WARN
org.apache.zookeeper.server.NIOServerCnxn: EndOfStreamException:
Unable to read additional data from client sessionid
0x13510f94d2d002c, likely client has closed socket
2012-01-24 16:19:12,315 INFO
org.apache.zookeeper.server.NIOServerCnxn: Closed socket connection
for client /127.0.0.1:41807 which had sessionid 0x13510f94d2d002c


Hadoop Information and Errors:
12/01/24 16:17:35 INFO zookeeper.ZooKeeper: Client
environment:java.library.path=/usr/local/hadoop-0.20.205.0/libexec/../lib
12/01/24 16:17:35 INFO zookeeper.ZooKeeper: Client
environment:java.io.tmpdir=/tmp
12/01/24 16:17:35 INFO zookeeper.ZooKeeper: Client
environment:java.compiler=<NA>
12/01/24 16:17:35 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
12/01/24 16:17:35 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
12/01/24 16:17:35 INFO zookeeper.ZooKeeper: Client
environment:os.version=2.6.35.6-48.fc14.x86_64
12/01/24 16:17:35 INFO zookeeper.ZooKeeper: Client environment:user.name=root
12/01/24 16:17:35 INFO zookeeper.ZooKeeper: Client environment:user.home=/root
12/01/24 16:17:35 INFO zookeeper.ZooKeeper: Client
environment:user.dir=/root/test/Alidade2
12/01/24 16:17:35 INFO zookeeper.ZooKeeper: Initiating client
connection, connectString=localhost:2181 sessionTimeout=180000
watcher=hconnection
12/01/24 16:17:35 INFO zookeeper.ClientCnxn: Opening socket connection
to server localhost/127.0.0.1:2181
12/01/24 16:17:35 INFO zookeeper.ClientCnxn: Socket connection
established to localhost/127.0.0.1:2181, initiating session
12/01/24 16:17:35 INFO zookeeper.ClientCnxn: Session establishment
complete on server localhost/127.0.0.1:2181, sessionid =
0x13510f94d2d002c, negotiated timeout = 180000
Submitting job
12/01/24 16:17:37 INFO mapred.JobClient: Running job: job_201201241325_0027
12/01/24 16:17:39 INFO mapred.JobClient:  map 0% reduce 0%
12/01/24 16:18:03 INFO mapred.JobClient: Task Id :
attempt_201201241325_0027_m_000000_0, Status : FAILED
org.apache.hadoop.hbase.ZooKeeperConnectionException: HBase is able to
connect to ZooKeeper but the connection closes immediately. This could
be a sign that the server has too many connections (30 is the
default). Consider inspecting your ZK server logs for that error and
then make sure you are reusing HBaseConfiguration as often as you can.
See HTable's javadoc for more information.
	at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:155)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:1002)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.setupZookeeperTrackers(HConnectionManager.java:304)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:295)
	at org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:157)
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:169)
	at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:147)
	at Common.DB.LandmarkDB.<init>(LandmarkDB.java:73)
	at Common.DB.LandmarkDB.getInstance(LandmarkDB.java:54)
	at Common.Data.ObservationRecord.init(ObservationRecord.java:301)
	at pMapReduce.Map.RawInputMapper.setup(RawInputMapper.java:45)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
	at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for /hbase
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:809)
	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:837)
	at org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndFailSilent(ZKUtil.java:903)
	at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:133)
	... 18 more

attempt_201201241325_0027_m_000000_0:  INIT: We are starting with
85602712 bytes free, 416284672 bytes total
attempt_201201241325_0027_m_000000_0: Subnet Bit len is 0

On Tue, Jan 24, 2012 at 3:10 PM, Fei Dong <dongfeiwww@gmail.com> wrote:
> Hi Stack
>
> On Tue, Jan 24, 2012 at 12:11 PM, Stack <stack@duboce.net> wrote:
>> On Tue, Jan 24, 2012 at 8:11 AM, Fei Dong <dongfeiwww@gmail.com> wrote:
>>> It says only Hadoop 0.20.205.x can match?
>>>
>>
>> No.  Also includes 1.0.0 and other hadoop offerings (read through that section)
>>
>>
>>
>>> I did not run any application before, so it should not have concurrent
>>> problem. Then I set it in hbase-site.xml, it still reports such error.
>>>    <name>hbase.zookeeper.property.maxClientCnxns</name>
>>>    <value>1000</value>
>>>
>>
>> Anything in zk logs?  If you connect to it w/ zkcli does it say > 1000
>> connections?
>>
> I test the hbase shell on another machine, which can "put", "get"
> record successfully. So I guess Zookeeper is running.
>
>>
>
> It is weird that it does not mention any path or class name behind
> "NoClassDefFoundError"
> It seems some error occurs when copying jar from JobTracker to
> TaskTracker, or it does not copy.
>
> The task tracker error log:
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> Caused by: java.lang.ClassNotFoundException:
>        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> Could not find the main class: .  Program will exit.
>>
>> St.Ack
>
>
>
> --
> Best Regards,
> --
> Fei Dong



-- 
Best Regards,
--
Fei Dong

Mime
View raw message