hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ikapo...@csd.auth.gr
Subject Re: Aggregator problem
Date Mon, 24 Feb 2014 15:26:06 GMT
Ok I will check again. I just found it strange to throw that kind of  
error without changing anything.



Quoting "Edward J. Yoon" <edwardyoon@apache.org>:

> Please see  
> http://svn.apache.org/repos/asf/hama/trunk/examples/src/main/java/org/apache/hama/examples/PageRank.java
> and check your program whether it is implemented correctly or not.
>
> I guess, it's not a problem of aggregator. It's a bug in your program.
>
> On Mon, Feb 24, 2014 at 6:50 PM,  <ikapoura@csd.auth.gr> wrote:
>> This is the code. What changed and suddenly it throws the error?
>>
>>
>>
>> package corever2;
>> import java.io.DataInput;
>> import java.io.DataOutput;
>> import java.io.IOException;
>> import org.apache.hadoop.io.DoubleWritable;
>> import org.apache.hadoop.io.IntWritable;
>> import org.apache.hadoop.io.Text;
>> import org.apache.hadoop.io.Writable;
>>
>> public class MessageCore implements Writable {
>>
>>     private DoubleWritable type;
>>     private Text nodeID;
>>     private IntWritable degree;
>>     private IntWritable maxDegree;
>>     private IntWritable nodeCounter;
>>
>>     public MessageCore() {
>>         type = new DoubleWritable();
>>         nodeID = new Text();
>>         degree = new IntWritable();
>>         maxDegree = new IntWritable();
>>         nodeCounter = new IntWritable();
>>     }
>>
>>     public MessageCore(DoubleWritable type, Text nodeID, IntWritable degree,
>> IntWritable maxDegree, IntWritable nodeCounter) {
>>         this.type = type;
>>         this.nodeID = nodeID;
>>         this.degree = degree;
>>         this.maxDegree = maxDegree;
>>         this.nodeCounter = nodeCounter;
>>     }
>>
>>     public MessageCore(DoubleWritable type, Text nodeID) {
>>         this.type = type;
>>         this.nodeID = nodeID;
>>         degree = new IntWritable();
>>         maxDegree = new IntWritable();
>>         nodeCounter = new IntWritable();
>>     }
>>
>>     public void setNodeCounter(IntWritable nodeCounter) {
>>         this.nodeCounter = nodeCounter;
>>     }
>>
>>     public void setMaxDegree(IntWritable maxDegree) {
>>         this.maxDegree = maxDegree;
>>     }
>>
>>     public void setType(DoubleWritable type) {
>>         this.type = type;
>>     }
>>
>>     public void setNodeID(Text nodeID) {
>>         this.nodeID = nodeID;
>>     }
>>
>>     public DoubleWritable getType() {
>>         return type;
>>     }
>>
>>     public Text getNodeID() {
>>         return nodeID;
>>     }
>>
>>     public IntWritable getDegree() {
>>         return degree;
>>     }
>>
>>     public IntWritable getMaxDegree() {
>>         return maxDegree;
>>     }
>>
>>     public IntWritable getNodeCounter() {
>>         return nodeCounter;
>>     }
>>
>>     public void setDegree(IntWritable degree) {
>>         this.degree = degree;
>>     }
>>
>>     @Override
>>     public String toString() {
>>         return nodeID + "\t" + degree;
>>     }
>>
>>     @Override
>>     public void write(DataOutput d) throws IOException {
>>         type.write(d);
>>         nodeID.write(d);
>>         degree.write(d);
>>         maxDegree.write(d);
>>         nodeCounter.write(d);
>>     }
>>
>>     @Override
>>     public void readFields(DataInput di) throws  IOException {
>>         type.readFields(di);
>>         nodeID.readFields(di);
>>         degree.readFields(di);
>>         maxDegree.readFields(di);
>>         nodeCounter.readFields(di);
>>
>>     }
>> }
>>
>>
>>
>> Quoting "Edward J. Yoon" <edwardyoon@apache.org>:
>>
>>> Please attach your MessageCore.
>>>
>>> On Sun, Feb 23, 2014 at 11:37 PM,  <ikapoura@csd.auth.gr> wrote:
>>>>
>>>> Hello there,
>>>>
>>>> i downloaded hama-trunk a few days ago and till now I am trying to
>>>> resolve
>>>> the following problem that came up:
>>>>
>>>> My program runs without errors without aggregators. When I set the
>>>> aggregator class to be used, I have the following error (which is like
>>>> "cant
>>>> find the message class" when the java file exists in the package!):
>>>>
>>>> corever2 is the package and MessageCore is the name of the class.
>>>>
>>>> java.io.IOException: can't find class: corever2.MessageCore because
>>>> corever2.MessageCore
>>>>         at
>>>>
>>>> org.apache.hadoop.io.AbstractMapWritable.readFields(AbstractMapWritable.java:204)
>>>>         at
>>>> org.apache.hadoop.io.MapWritable.readFields(MapWritable.java:146)
>>>>         at
>>>>
>>>> org.apache.hama.graph.GraphJobMessage.readFields(GraphJobMessage.java:129)
>>>>         at
>>>> org.apache.hama.bsp.BSPMessageBundle$1.next(BSPMessageBundle.java:108)
>>>>         at
>>>> org.apache.hama.bsp.BSPMessageBundle$1.next(BSPMessageBundle.java:78)
>>>>         at
>>>>
>>>> org.apache.hama.bsp.LocalBSPRunner$LocalMessageManager.transfer(LocalBSPRunner.java:356)
>>>>         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:381)
>>>>         at
>>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:133)
>>>>         at
>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:258)
>>>>         at
>>>>
>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:288)
>>>>         at
>>>>
>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:212)
>>>>         at
>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>         at
>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>>>         at
>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>         at
>>>>
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>>>         at
>>>>
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>>>         at java.lang.Thread.run(Thread.java:662)
>>>> java.io.EOFException
>>>>         at java.io.DataInputStream.readFully(DataInputStream.java:180)
>>>>         at org.apache.hadoop.io.Text.readFields(Text.java:265)
>>>>         at
>>>>
>>>> org.apache.hama.graph.GraphJobMessage.readFields(GraphJobMessage.java:136)
>>>>         at
>>>> org.apache.hama.bsp.BSPMessageBundle$1.next(BSPMessageBundle.java:108)
>>>>         at
>>>> org.apache.hama.bsp.BSPMessageBundle$1.next(BSPMessageBundle.java:78)
>>>>         at
>>>>
>>>> org.apache.hama.bsp.LocalBSPRunner$LocalMessageManager.transfer(LocalBSPRunner.java:356)
>>>>         at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:381)
>>>>         at
>>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:133)
>>>>         at
>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:258)
>>>>         at
>>>>
>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:288)
>>>>         at
>>>>
>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:212)
>>>>         at
>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>         at
>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>>>         at
>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>         at
>>>>
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>>>         at
>>>>
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>>>         at java.lang.Thread.run(Thread.java:662)
>>>> 14/02/23 18:30:23 ERROR bsp.LocalBSPRunner: Exception during BSP
>>>> execution!
>>>> java.lang.UnsupportedOperationException: Unknown message type:
>>>> GraphJobMessage [flag=0, map=null, vertexId=, vertexValue=null]
>>>>         at
>>>>
>>>> org.apache.hama.graph.GraphJobRunner.parseMessages(GraphJobRunner.java:590)
>>>>         at
>>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:136)
>>>>         at
>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:258)
>>>>         at
>>>>
>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:288)
>>>>         at
>>>>
>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:212)
>>>>         at
>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>         at
>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>>>         at
>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>         at
>>>>
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>>>         at
>>>>
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>>>         at java.lang.Thread.run(Thread.java:662)
>>>> 14/02/23 18:30:23 ERROR bsp.LocalBSPRunner: Exception during BSP
>>>> execution!
>>>> java.util.concurrent.ExecutionException:
>>>> java.lang.UnsupportedOperationException: Unknown message type:
>>>> GraphJobMessage [flag=0, map=null, vertexId=, vertexValue=null]
>>>>         at
>>>> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>>>>         at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>>>         at
>>>>
>>>> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:315)
>>>>         at java.lang.Thread.run(Thread.java:662)
>>>> Caused by: java.lang.UnsupportedOperationException: Unknown message type:
>>>> GraphJobMessage [flag=0, map=null, vertexId=, vertexValue=null]
>>>>         at
>>>>
>>>> org.apache.hama.graph.GraphJobRunner.parseMessages(GraphJobRunner.java:590)
>>>>         at
>>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:136)
>>>>         at
>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:258)
>>>>         at
>>>>
>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:288)
>>>>         at
>>>>
>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:212)
>>>>         at
>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>         at
>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>>>         at
>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>         at
>>>>
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>>>         at
>>>>
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>>>         ... 1 more
>>>> 14/02/23 18:30:25 INFO bsp.BSPJobClient: Current supersteps number: 1
>>>> 14/02/23 18:30:25 INFO bsp.BSPJobClient: Job failed.
>>>>
>>>>
>>>> The only thing I managed to figure is that this comes up when the
>>>> aggregators are enabled.
>>>>
>>>> Thank you.
>>>>
>>>
>>>
>>>
>>> --
>>> Edward J. Yoon (@eddieyoon)
>>> Chief Executive Officer
>>> DataSayer, Inc.
>>
>>
>>
>
>
>
> --
> Edward J. Yoon (@eddieyoon)
> Chief Executive Officer
> DataSayer, Inc.




Mime
View raw message