hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edward.y...@samsung.com>
Subject RE: RE: Is This a Bug of Hama?
Date Thu, 28 Apr 2016 06:20:23 GMT
I doubt you have an orphaned or dangling links. For example, vertex a has 
edges b, c, d. The vertex b and c are exist in your input graph but c doesn't. 
Then, the framework will try to find and compute c vertex with received 
messages but it's null.

Please try to set "hama.graph.self.ref" to true in your job configuration.

conf.getBoolean("hama.graph.self.ref", true);

--
Best Regards, Edward J. Yoon

-----Original Message-----
From: 步?云 [mailto:mailliuping@qq.com]
Sent: Thursday, April 28, 2016 12:15 PM
To: user
Subject: Re:RE: Is This a Bug of Hama?

Hi,
    Thank you for your reply. TriTextPair is wirtable and the code is as 
follow.
    I don't know whether there is problem in TriTextPair. I re-write TextPair 
in hama examples to TriTextPair.
    I'm really confused wht sometime the job run successfully and other time 
failed. I need help.
   Thanks a lot.


public final class TriTextPair implements Writable {
	Text first;
	Text second;
	Text third;


	public TriTextPair() {
		first = new Text("");
		second = new Text("");
		third = new Text("");
		fourth = new Text("");
	}

	public TriTextPair(Text first, Text second, Text third, Text fourth) {
		this.first = first;
		this.second = second;
		this.third = third;
		this.fourth = fourth;
	}

	public Text getFirst() {
		return first;
	}


	public void setFirst(Text first) {
		this.first = first;
	}


	public Text getSecond() {
		return second;
	}


	public void setSecond(Text second) {
		this.second = second;
	}


	public Text getThird() {
		return third;
	}


	public void setThird(Text third) {
		this.third = third;
	}


	@Override
	public void write(DataOutput out) throws IOException {
		first.write(out);
		second.write(out);
		third.write(out);
	}


	@Override
	public void readFields(DataInput in) throws IOException {


		first.readFields(in);
		second.readFields(in);
		third.readFields(in);
	}}



------------------ Original ------------------
From:  "Edward J. Yoon";<edward.yoon@samsung.com>;
Date:  Apr 28, 2016
To:  "user"<user@hama.apache.org>;

Subject:  RE: Is This a Bug of Hama?



Hello,

Please check whether TriTextPair is writable. Please attach that code too if
possible.

--
Best Regards, Edward J. Yoon


-----Original Message-----
From: 꼍행暾 [mailto:mailliuping@qq.com]
Sent: Wednesday, April 27, 2016 8:11 PM
To: user
Subject: Is This a Bug of Hama?

Hello,
     I'm using hama-0.7.0 to do my experiment. While I had some problems.
For the same program and the same inputs, sometimes I could run the program
successfully. But sometimes some problems occured, and then the job failed.
     I would show my code before error message. My code is as follow:


public static class ProbMatchVertex extends Vertex<Text, NullWritable,
TriTextPair> {
		private String parents = new String("");

		@Override
		public void compute(Iterable<TriTextPair> messages) throws
IOException {

			if (getSuperstepCount() == 0) {
				sendMessageToNeighbors(new
TriTextPair(getVertexID(), getVertexLabel(), new Text(""), new Text(""), new
Text()));
			} else if (getSuperstepCount() == 1) {
				for (TriTextPair msg : messages) {
					parents += msg.toString() + "|";
					sendMessage(msg.getFirst(), new
TriTextPair(getVertexID(), getVertexLabel(), new Text(""), new Text(""), new
Text()));
				}
			} else if (getSuperstepCount() == 2) {
				List<TriTextPair> parentList =
str2TriList(parents);
				for (int i=0;i<parentList.size();i++) {

sendMessage(parentList.get(i).getFirst(),
							new
TriTextPair(getVertexID(), new Text("frC"), new Text(matchMapStr), new
Text(loseMapStr), getVertexID()));
				}
			} else if (getSuperstepCount() > 2) {
				......
			}
		}



                 @Override
		public void writeState(DataOutput out) throws IOException {
		    out.writeUTF(parents);
		  }

		@Override
		  public void readState(DataInput in) throws IOException {
		    parents = in.readUTF();
		  }
}
         And the error message is as follow:
INFO message.HamaMessageManagerImpl: BSPPeer address:slave2.novalocal
port:61001
attempt_201604270849_0010_000014_0: 16/04/27 10:53:52 INFO
sync.ZKSyncClient: Initializing ZK Sync Client
attempt_201604270849_0010_000014_0: 16/04/27 10:53:52 INFO
sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At
slave2.novalocal/10.0.2.98:61001
attempt_201604270849_0010_000014_0: 16/04/27 10:53:52 INFO
sync.ZooKeeperSyncClientImpl: TaskAttemptID :
attempt_201604270849_0010_000014_0
attempt_201604270849_0010_000014_0: 16/04/27 10:54:13 INFO
graph.GraphJobRunner: 55884 vertices are loaded into slave2.novalocal:61001
attempt_201604270849_0010_000014_0: 16/04/27 10:54:13 INFO
graph.GraphJobRunner: Total time spent for loading vertices: 8860 ms
attempt_201604270849_0010_000014_0: 16/04/27 10:54:14 INFO
graph.GraphJobRunner: Total time spent for broadcasting global vertex count:
349 ms
attempt_201604270849_0010_000014_0: 16/04/27 10:54:21 INFO
graph.GraphJobRunner: Total time spent for initial superstep: 6937 ms
attempt_201604270849_0010_000014_0: 16/04/27 10:54:33 INFO
graph.GraphJobRunner: Total time spent for broadcasting aggregation values:
276 ms
attempt_201604270849_0010_000014_0: 16/04/27 10:54:33 ERROR bsp.BSPTask:
Error running bsp setup and bsp function.
attempt_201604270849_0010_000014_0: java.lang.NullPointerException
attempt_201604270849_0010_000014_0:     at
org.apache.hama.util.UnsafeByteArrayInputStream.<init>(UnsafeByteArrayInputS
tream.java:63)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.util.WritableUtils.unsafeDeserialize(WritableUtils.java:63)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.MapVerticesInfo.get(MapVerticesInfo.java:101)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.GraphJobRunner$ComputeRunnable.<init>(GraphJobRunner.j
ava:359)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.GraphJobRunner.doSuperstep(GraphJobRunner.java:270)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:182)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:171)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.bsp.BSPTask.run(BSPTask.java:144)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1255)
attempt_201604270849_0010_000014_0: 16/04/27 10:54:34 INFO ipc.Server:
Stopping server on 61001
attempt_201604270849_0010_000014_0: 16/04/27 10:54:34 INFO ipc.Server: IPC
Server handler 0 on 61001: exiting
attempt_201604270849_0010_000014_0: 16/04/27 10:54:34 INFO ipc.Server: IPC
Server handler 1 on 61001: exiting
attempt_201604270849_0010_000014_0: 16/04/27 10:54:34 INFO ipc.Server:
Stopping IPC Server listener on 61001
attempt_201604270849_0010_000014_0: 16/04/27 10:54:34 INFO ipc.Server: IPC
Server handler 4 on 61001: exiting
attempt_201604270849_0010_000014_0: 16/04/27 10:54:34 INFO ipc.Server: IPC
Server handler 3 on 61001: exiting
attempt_201604270849_0010_000014_0: 16/04/27 10:54:34 INFO
Configuration.deprecation: mapred.cache.localFiles is deprecated. Instead,
use mapreduce.job.cache.local.files
attempt_201604270849_0010_000014_0: 16/04/27 10:54:34 INFO ipc.Server: IPC
Server handler 2 on 61001: exiting
attempt_201604270849_0010_000014_0: 16/04/27 10:54:34 ERROR bsp.BSPTask:
Shutting down ping service.
attempt_201604270849_0010_000014_0: 16/04/27 10:54:34 INFO ipc.Server:
Stopping IPC Server Responder
attempt_201604270849_0010_000014_0: 16/04/27 10:54:34 FATAL bsp.GroomServer:
Error running child
attempt_201604270849_0010_000014_0: java.lang.NullPointerException
attempt_201604270849_0010_000014_0:     at
org.apache.hama.util.UnsafeByteArrayInputStream.<init>(UnsafeByteArrayInputS
tream.java:63)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.util.WritableUtils.unsafeDeserialize(WritableUtils.java:63)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.MapVerticesInfo.get(MapVerticesInfo.java:101)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.GraphJobRunner$ComputeRunnable.<init>(GraphJobRunner.j
ava:359)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.GraphJobRunner.doSuperstep(GraphJobRunner.java:270)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:182)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:171)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.bsp.BSPTask.run(BSPTask.java:144)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1255)
attempt_201604270849_0010_000014_0: java.lang.NullPointerException
attempt_201604270849_0010_000014_0:     at
org.apache.hama.util.UnsafeByteArrayInputStream.<init>(UnsafeByteArrayInputS
tream.java:63)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.util.WritableUtils.unsafeDeserialize(WritableUtils.java:63)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.MapVerticesInfo.get(MapVerticesInfo.java:101)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.GraphJobRunner$ComputeRunnable.<init>(GraphJobRunner.j
ava:359)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.GraphJobRunner.doSuperstep(GraphJobRunner.java:270)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:182)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:171)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.bsp.BSPTask.run(BSPTask.java:144)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1255)
16/04/27 10:55:39 INFO bsp.BSPJobClient: Job failed.



         I really don't know how this problem occured. Who can help me?
Thanks a lot.


Ping Liu
Sincerely



Mime
View raw message