hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lyu Xuedong <lxd.1...@gmail.com>
Subject Why my distributed mode does not work?
Date Tue, 16 Apr 2013 13:55:21 GMT
My project can work well in eclipse, but when I export it as a jar file 
and submit it to a two-node hama cluster, some errors and fatal occur if 
the input file is over 64M.
I suspect that my configure files of hadoop or hama is somewhere not 
right, but a pi estimator can run normally on my cluster and 
GroomServer$BSPPerChild can be seen on each nodes. I debug for a whole 
day, nothing improved.
What in my input file are a large number of RDF triples: "<subject> 
<predicate> <object> ." My task is to create vertices for subjects and 
objects. Predicates are subjects' edges.
What should I do ?

Terminal output:

13/04/16 21:13:36 INFO bgp.HamaBgpComplete: Job begain.
13/04/16 21:13:37 INFO bsp.FileInputFormat: Total input paths to process : 2
13/04/16 21:13:38 INFO bsp.BSPJobClient: Running job: job_201304161357_0015
13/04/16 21:13:41 INFO bsp.BSPJobClient: Current supersteps number: 0
13/04/16 21:13:47 INFO bsp.BSPJobClient: Current supersteps number: 2
13/04/16 21:13:53 INFO bsp.BSPJobClient: Current supersteps number: 3
13/04/16 21:13:59 INFO bsp.BSPJobClient: Current supersteps number: 4
13/04/16 21:14:05 INFO bsp.BSPJobClient: Current supersteps number: 5
attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO 
sync.ZKSyncClient: Initializing ZK Sync Client
attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO 
sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At 
hadoop1/1.2.3.4:61002
attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: 
Starting SocketReader
attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: 
IPC Server Responder: starting
attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: 
IPC Server handler 0 on 61002: starting
attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO 
message.HadoopMessageManagerImpl:  BSPPeer address:hadoop1 port:61002
attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: 
IPC Server listener on 61002: starting
attempt_201304161357_0015_000000_0: 13/04/16 21:14:03 ERROR bsp.BSPTask: 
Error running bsp setup and bsp function.
attempt_201304161357_0015_000000_0: java.lang.NullPointerException
attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server: 
Stopping server on 61002
attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server: 
IPC Server handler 0 on 61002: exiting
attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server: 
Stopping IPC Server listener on 61002
attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server: 
Stopping IPC Server Responder
attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO 
metrics.RpcInstrumentation: shut down
attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 ERROR bsp.BSPTask: 
Shutting down ping service.
attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 FATAL 
bsp.GroomServer: Error running child
attempt_201304161357_0015_000000_0: java.lang.NullPointerException
attempt_201304161357_0015_000000_0: java.lang.NullPointerException
13/04/16 21:15:11 INFO bsp.BSPJobClient: Job failed.

tasklogs:
attempt_201304161357_0015_000000_0.log
13/04/16 21:13:43 INFO sync.ZKSyncClient: Initializing ZK Sync Client
13/04/16 21:13:43 INFO sync.ZooKeeperSyncClientImpl: Start connecting to 
Zookeeper! At hadoop1/1.2.3.4:61002
13/04/16 21:13:43 INFO ipc.Server: Starting SocketReader
13/04/16 21:13:43 INFO ipc.Server: IPC Server Responder: starting
13/04/16 21:13:43 INFO ipc.Server: IPC Server handler 0 on 61002: starting
13/04/16 21:13:43 INFO message.HadoopMessageManagerImpl:  BSPPeer 
address:hadoop1 port:61002
13/04/16 21:13:43 INFO ipc.Server: IPC Server listener on 61002: starting
13/04/16 21:14:03 ERROR bsp.BSPTask: Error running bsp setup and bsp 
function.
java.lang.NullPointerException
13/04/16 21:14:04 INFO ipc.Server: Stopping server on 61002
13/04/16 21:14:04 INFO ipc.Server: IPC Server handler 0 on 61002: exiting
13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server listener on 61002
13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server Responder
13/04/16 21:14:04 INFO metrics.RpcInstrumentation: shut down
13/04/16 21:14:04 ERROR bsp.BSPTask: Shutting down ping service.
13/04/16 21:14:04 FATAL bsp.GroomServer: Error running child
java.lang.NullPointerException
java.lang.NullPointerException

attempt_201304161357_0015_000001_0.log
13/04/16 21:13:42 INFO sync.ZKSyncClient: Initializing ZK Sync Client
13/04/16 21:13:42 INFO sync.ZooKeeperSyncClientImpl: Start connecting to 
Zookeeper! At hadoop1/1.2.3.4:61001
13/04/16 21:13:42 ERROR sync.ZooKeeperSyncClientImpl: 
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = 
NoNode for /bsp/job_201304161357_0015/peers
13/04/16 21:13:42 INFO ipc.Server: Starting SocketReader
13/04/16 21:13:42 INFO ipc.Server: IPC Server Responder: starting
13/04/16 21:13:42 INFO message.HadoopMessageManagerImpl:  BSPPeer 
address:hadoop1 port:61001
13/04/16 21:13:42 INFO ipc.Server: IPC Server listener on 61001: starting
13/04/16 21:13:42 INFO ipc.Server: IPC Server handler 0 on 61001: starting
13/04/16 21:14:06 ERROR bsp.BSPPeerImpl: Error while sending messages
java.io.IOException: Call to hadoop1/1.2.3.4:61002 failed on local 
exception: java.io.EOFException
     at org.apache.hadoop.ipc.Client.wrapException(Client.java:1103)
     at org.apache.hadoop.ipc.Client.call(Client.java:1071)
     at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
     at $Proxy3.put(Unknown Source)
     at 
org.apache.hama.bsp.message.HadoopMessageManagerImpl.transfer(HadoopMessageManagerImpl.java:108)
     at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:410)
     at org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:118)
     at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166)
     at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143)
     at 
org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1266)
Caused by: java.io.EOFException
     at java.io.DataInputStream.readInt(DataInputStream.java:375)
     at 
org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:800)
     at org.apache.hadoop.ipc.Client$Connection.run(Client.java:745)










Mime
View raw message