hama: 0.6.0 hadoop : 1.0.4 JDK : 1.6 OS : ubuntu 12.04 On 04/17/2013 05:47 AM, Edward J. Yoon wrote: > Your version? > > On Wed, Apr 17, 2013 at 12:07 AM, Lvxuedong wrote: >> Hi, Edward, thank you. But your suggestion seems do not work, do you have some other advice ? >> >> Is java.lang.NullPointerException related with heap size ? >> >> >> >> 在 2013-4-16，22:13，"Edward J. Yoon" 写道： >> >>> I guess you need to increase the child processor JVM heap size. >>> >>> - conf/hama-site.xml: >>> >>> >>> bsp.child.java.opts >>> -Xmx2048m >>> >>> >>> On Tue, Apr 16, 2013 at 10:55 PM, Lyu Xuedong wrote: >>>> My project can work well in eclipse, but when I export it as a jar file and >>>> submit it to a two-node hama cluster, some errors and fatal occur if the >>>> input file is over 64M. >>>> I suspect that my configure files of hadoop or hama is somewhere not right, >>>> but a pi estimator can run normally on my cluster and >>>> GroomServer$BSPPerChild can be seen on each nodes. I debug for a whole day, >>>> nothing improved. >>>> What in my input file are a large number of RDF triples: " >>>> ." My task is to create vertices for subjects and >>>> objects. Predicates are subjects' edges. >>>> What should I do ? >>>> >>>> Terminal output: >>>> >>>> 13/04/16 21:13:36 INFO bgp.HamaBgpComplete: Job begain. >>>> 13/04/16 21:13:37 INFO bsp.FileInputFormat: Total input paths to process : 2 >>>> 13/04/16 21:13:38 INFO bsp.BSPJobClient: Running job: job_201304161357_0015 >>>> 13/04/16 21:13:41 INFO bsp.BSPJobClient: Current supersteps number: 0 >>>> 13/04/16 21:13:47 INFO bsp.BSPJobClient: Current supersteps number: 2 >>>> 13/04/16 21:13:53 INFO bsp.BSPJobClient: Current supersteps number: 3 >>>> 13/04/16 21:13:59 INFO bsp.BSPJobClient: Current supersteps number: 4 >>>> 13/04/16 21:14:05 INFO bsp.BSPJobClient: Current supersteps number: 5 >>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO >>>> sync.ZKSyncClient: Initializing ZK Sync Client >>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO >>>> sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At >>>> hadoop1/1.2.3.4:61002 >>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: >>>> Starting SocketReader >>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: IPC >>>> Server Responder: starting >>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: IPC >>>> Server handler 0 on 61002: starting >>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO >>>> message.HadoopMessageManagerImpl: BSPPeer address:hadoop1 port:61002 >>>> attempt_201304161357_0015_000000_0: 13/04/16 21:13:43 INFO ipc.Server: IPC >>>> Server listener on 61002: starting >>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:03 ERROR bsp.BSPTask: >>>> Error running bsp setup and bsp function. >>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException >>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server: >>>> Stopping server on 61002 >>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server: IPC >>>> Server handler 0 on 61002: exiting >>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server: >>>> Stopping IPC Server listener on 61002 >>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO ipc.Server: >>>> Stopping IPC Server Responder >>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 INFO >>>> metrics.RpcInstrumentation: shut down >>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 ERROR bsp.BSPTask: >>>> Shutting down ping service. >>>> attempt_201304161357_0015_000000_0: 13/04/16 21:14:04 FATAL bsp.GroomServer: >>>> Error running child >>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException >>>> attempt_201304161357_0015_000000_0: java.lang.NullPointerException >>>> 13/04/16 21:15:11 INFO bsp.BSPJobClient: Job failed. >>>> >>>> tasklogs: >>>> attempt_201304161357_0015_000000_0.log >>>> 13/04/16 21:13:43 INFO sync.ZKSyncClient: Initializing ZK Sync Client >>>> 13/04/16 21:13:43 INFO sync.ZooKeeperSyncClientImpl: Start connecting to >>>> Zookeeper! At hadoop1/1.2.3.4:61002 >>>> 13/04/16 21:13:43 INFO ipc.Server: Starting SocketReader >>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server Responder: starting >>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server handler 0 on 61002: starting >>>> 13/04/16 21:13:43 INFO message.HadoopMessageManagerImpl: BSPPeer >>>> address:hadoop1 port:61002 >>>> 13/04/16 21:13:43 INFO ipc.Server: IPC Server listener on 61002: starting >>>> 13/04/16 21:14:03 ERROR bsp.BSPTask: Error running bsp setup and bsp >>>> function. >>>> java.lang.NullPointerException >>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping server on 61002 >>>> 13/04/16 21:14:04 INFO ipc.Server: IPC Server handler 0 on 61002: exiting >>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server listener on 61002 >>>> 13/04/16 21:14:04 INFO ipc.Server: Stopping IPC Server Responder >>>> 13/04/16 21:14:04 INFO metrics.RpcInstrumentation: shut down >>>> 13/04/16 21:14:04 ERROR bsp.BSPTask: Shutting down ping service. >>>> 13/04/16 21:14:04 FATAL bsp.GroomServer: Error running child >>>> java.lang.NullPointerException >>>> java.lang.NullPointerException >>>> >>>> attempt_201304161357_0015_000001_0.log >>>> 13/04/16 21:13:42 INFO sync.ZKSyncClient: Initializing ZK Sync Client >>>> 13/04/16 21:13:42 INFO sync.ZooKeeperSyncClientImpl: Start connecting to >>>> Zookeeper! At hadoop1/1.2.3.4:61001 >>>> 13/04/16 21:13:42 ERROR sync.ZooKeeperSyncClientImpl: >>>> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = >>>> NoNode for /bsp/job_201304161357_0015/peers >>>> 13/04/16 21:13:42 INFO ipc.Server: Starting SocketReader >>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server Responder: starting >>>> 13/04/16 21:13:42 INFO message.HadoopMessageManagerImpl: BSPPeer >>>> address:hadoop1 port:61001 >>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server listener on 61001: starting >>>> 13/04/16 21:13:42 INFO ipc.Server: IPC Server handler 0 on 61001: starting >>>> 13/04/16 21:14:06 ERROR bsp.BSPPeerImpl: Error while sending messages >>>> java.io.IOException: Call to hadoop1/1.2.3.4:61002 failed on local >>>> exception: java.io.EOFException >>>> at org.apache.hadoop.ipc.Client.wrapException(Client.java:1103) >>>> at org.apache.hadoop.ipc.Client.call(Client.java:1071) >>>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225) >>>> at $Proxy3.put(Unknown Source) >>>> at >>>> org.apache.hama.bsp.message.HadoopMessageManagerImpl.transfer(HadoopMessageManagerImpl.java:108) >>>> at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:410) >>>> at org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:118) >>>> at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:166) >>>> at org.apache.hama.bsp.BSPTask.run(BSPTask.java:143) >>>> at >>>> org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1266) >>>> Caused by: java.io.EOFException >>>> at java.io.DataInputStream.readInt(DataInputStream.java:375) >>>> at >>>> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:800) >>>> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:745) >>> >>> >>> -- >>> Best Regards, Edward J. Yoon >>> @eddieyoon > >