hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ikapo...@csd.auth.gr
Subject Re: Aggregator problem
Date Tue, 25 Feb 2014 11:25:11 GMT

I really can't find a solution, can you please send me a simple custom  
message type of yours that is working so that I can clear some bug  
possibilities out?

Thanks.


Quoting ikapoura@csd.auth.gr:

> Ok I will check again. I just found it strange to throw that kind of  
> error without changing anything.
>
>
>
> Quoting "Edward J. Yoon" <edwardyoon@apache.org>:
>
>> Please see  
>> http://svn.apache.org/repos/asf/hama/trunk/examples/src/main/java/org/apache/hama/examples/PageRank.java
>> and check your program whether it is implemented correctly or not.
>>
>> I guess, it's not a problem of aggregator. It's a bug in your program.
>>
>> On Mon, Feb 24, 2014 at 6:50 PM,  <ikapoura@csd.auth.gr> wrote:
>>> This is the code. What changed and suddenly it throws the error?
>>>
>>>
>>>
>>> package corever2;
>>> import java.io.DataInput;
>>> import java.io.DataOutput;
>>> import java.io.IOException;
>>> import org.apache.hadoop.io.DoubleWritable;
>>> import org.apache.hadoop.io.IntWritable;
>>> import org.apache.hadoop.io.Text;
>>> import org.apache.hadoop.io.Writable;
>>>
>>> public class MessageCore implements Writable {
>>>
>>>    private DoubleWritable type;
>>>    private Text nodeID;
>>>    private IntWritable degree;
>>>    private IntWritable maxDegree;
>>>    private IntWritable nodeCounter;
>>>
>>>    public MessageCore() {
>>>        type = new DoubleWritable();
>>>        nodeID = new Text();
>>>        degree = new IntWritable();
>>>        maxDegree = new IntWritable();
>>>        nodeCounter = new IntWritable();
>>>    }
>>>
>>>    public MessageCore(DoubleWritable type, Text nodeID, IntWritable degree,
>>> IntWritable maxDegree, IntWritable nodeCounter) {
>>>        this.type = type;
>>>        this.nodeID = nodeID;
>>>        this.degree = degree;
>>>        this.maxDegree = maxDegree;
>>>        this.nodeCounter = nodeCounter;
>>>    }
>>>
>>>    public MessageCore(DoubleWritable type, Text nodeID) {
>>>        this.type = type;
>>>        this.nodeID = nodeID;
>>>        degree = new IntWritable();
>>>        maxDegree = new IntWritable();
>>>        nodeCounter = new IntWritable();
>>>    }
>>>
>>>    public void setNodeCounter(IntWritable nodeCounter) {
>>>        this.nodeCounter = nodeCounter;
>>>    }
>>>
>>>    public void setMaxDegree(IntWritable maxDegree) {
>>>        this.maxDegree = maxDegree;
>>>    }
>>>
>>>    public void setType(DoubleWritable type) {
>>>        this.type = type;
>>>    }
>>>
>>>    public void setNodeID(Text nodeID) {
>>>        this.nodeID = nodeID;
>>>    }
>>>
>>>    public DoubleWritable getType() {
>>>        return type;
>>>    }
>>>
>>>    public Text getNodeID() {
>>>        return nodeID;
>>>    }
>>>
>>>    public IntWritable getDegree() {
>>>        return degree;
>>>    }
>>>
>>>    public IntWritable getMaxDegree() {
>>>        return maxDegree;
>>>    }
>>>
>>>    public IntWritable getNodeCounter() {
>>>        return nodeCounter;
>>>    }
>>>
>>>    public void setDegree(IntWritable degree) {
>>>        this.degree = degree;
>>>    }
>>>
>>>    @Override
>>>    public String toString() {
>>>        return nodeID + "\t" + degree;
>>>    }
>>>
>>>    @Override
>>>    public void write(DataOutput d) throws IOException {
>>>        type.write(d);
>>>        nodeID.write(d);
>>>        degree.write(d);
>>>        maxDegree.write(d);
>>>        nodeCounter.write(d);
>>>    }
>>>
>>>    @Override
>>>    public void readFields(DataInput di) throws  IOException {
>>>        type.readFields(di);
>>>        nodeID.readFields(di);
>>>        degree.readFields(di);
>>>        maxDegree.readFields(di);
>>>        nodeCounter.readFields(di);
>>>
>>>    }
>>> }
>>>
>>>
>>>
>>> Quoting "Edward J. Yoon" <edwardyoon@apache.org>:
>>>
>>>> Please attach your MessageCore.
>>>>
>>>> On Sun, Feb 23, 2014 at 11:37 PM,  <ikapoura@csd.auth.gr> wrote:
>>>>>
>>>>> Hello there,
>>>>>
>>>>> i downloaded hama-trunk a few days ago and till now I am trying to
>>>>> resolve
>>>>> the following problem that came up:
>>>>>
>>>>> My program runs without errors without aggregators. When I set the
>>>>> aggregator class to be used, I have the following error (which is like
>>>>> "cant
>>>>> find the message class" when the java file exists in the package!):
>>>>>
>>>>> corever2 is the package and MessageCore is the name of the class.
>>>>>
>>>>> java.io.IOException: can't find class: corever2.MessageCore because
>>>>> corever2.MessageCore
>>>>>        at
>>>>>
>>>>> org.apache.hadoop.io.AbstractMapWritable.readFields(AbstractMapWritable.java:204)
>>>>>        at
>>>>> org.apache.hadoop.io.MapWritable.readFields(MapWritable.java:146)
>>>>>        at
>>>>>
>>>>> org.apache.hama.graph.GraphJobMessage.readFields(GraphJobMessage.java:129)
>>>>>        at
>>>>> org.apache.hama.bsp.BSPMessageBundle$1.next(BSPMessageBundle.java:108)
>>>>>        at
>>>>> org.apache.hama.bsp.BSPMessageBundle$1.next(BSPMessageBundle.java:78)
>>>>>        at
>>>>>
>>>>> org.apache.hama.bsp.LocalBSPRunner$LocalMessageManager.transfer(LocalBSPRunner.java:356)
>>>>>        at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:381)
>>>>>        at
>>>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:133)
>>>>>        at
>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:258)
>>>>>        at
>>>>>
>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:288)
>>>>>        at
>>>>>
>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:212)
>>>>>        at
>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>        at
>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>>>>        at
>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>        at
>>>>>
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>>>>        at
>>>>>
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>>>>        at java.lang.Thread.run(Thread.java:662)
>>>>> java.io.EOFException
>>>>>        at java.io.DataInputStream.readFully(DataInputStream.java:180)
>>>>>        at org.apache.hadoop.io.Text.readFields(Text.java:265)
>>>>>        at
>>>>>
>>>>> org.apache.hama.graph.GraphJobMessage.readFields(GraphJobMessage.java:136)
>>>>>        at
>>>>> org.apache.hama.bsp.BSPMessageBundle$1.next(BSPMessageBundle.java:108)
>>>>>        at
>>>>> org.apache.hama.bsp.BSPMessageBundle$1.next(BSPMessageBundle.java:78)
>>>>>        at
>>>>>
>>>>> org.apache.hama.bsp.LocalBSPRunner$LocalMessageManager.transfer(LocalBSPRunner.java:356)
>>>>>        at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:381)
>>>>>        at
>>>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:133)
>>>>>        at
>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:258)
>>>>>        at
>>>>>
>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:288)
>>>>>        at
>>>>>
>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:212)
>>>>>        at
>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>        at
>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>>>>        at
>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>        at
>>>>>
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>>>>        at
>>>>>
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>>>>        at java.lang.Thread.run(Thread.java:662)
>>>>> 14/02/23 18:30:23 ERROR bsp.LocalBSPRunner: Exception during BSP
>>>>> execution!
>>>>> java.lang.UnsupportedOperationException: Unknown message type:
>>>>> GraphJobMessage [flag=0, map=null, vertexId=, vertexValue=null]
>>>>>        at
>>>>>
>>>>> org.apache.hama.graph.GraphJobRunner.parseMessages(GraphJobRunner.java:590)
>>>>>        at
>>>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:136)
>>>>>        at
>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:258)
>>>>>        at
>>>>>
>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:288)
>>>>>        at
>>>>>
>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:212)
>>>>>        at
>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>        at
>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>>>>        at
>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>        at
>>>>>
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>>>>        at
>>>>>
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>>>>        at java.lang.Thread.run(Thread.java:662)
>>>>> 14/02/23 18:30:23 ERROR bsp.LocalBSPRunner: Exception during BSP
>>>>> execution!
>>>>> java.util.concurrent.ExecutionException:
>>>>> java.lang.UnsupportedOperationException: Unknown message type:
>>>>> GraphJobMessage [flag=0, map=null, vertexId=, vertexValue=null]
>>>>>        at
>>>>> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>>>>>        at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>>>>        at
>>>>>
>>>>> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:315)
>>>>>        at java.lang.Thread.run(Thread.java:662)
>>>>> Caused by: java.lang.UnsupportedOperationException: Unknown message type:
>>>>> GraphJobMessage [flag=0, map=null, vertexId=, vertexValue=null]
>>>>>        at
>>>>>
>>>>> org.apache.hama.graph.GraphJobRunner.parseMessages(GraphJobRunner.java:590)
>>>>>        at
>>>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:136)
>>>>>        at
>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:258)
>>>>>        at
>>>>>
>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:288)
>>>>>        at
>>>>>
>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:212)
>>>>>        at
>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>        at
>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>>>>        at
>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>        at
>>>>>
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>>>>        at
>>>>>
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>>>>        ... 1 more
>>>>> 14/02/23 18:30:25 INFO bsp.BSPJobClient: Current supersteps number: 1
>>>>> 14/02/23 18:30:25 INFO bsp.BSPJobClient: Job failed.
>>>>>
>>>>>
>>>>> The only thing I managed to figure is that this comes up when the
>>>>> aggregators are enabled.
>>>>>
>>>>> Thank you.
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Edward J. Yoon (@eddieyoon)
>>>> Chief Executive Officer
>>>> DataSayer, Inc.
>>>
>>>
>>>
>>
>>
>>
>> --
>> Edward J. Yoon (@eddieyoon)
>> Chief Executive Officer
>> DataSayer, Inc.




Mime
View raw message