hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: Aggregator problem
Date Tue, 25 Feb 2014 12:51:54 GMT
See http://people.apache.org/~edwardyoon/MyTest.txt

On Tue, Feb 25, 2014 at 8:25 PM,  <ikapoura@csd.auth.gr> wrote:
>
> I really can't find a solution, can you please send me a simple custom
> message type of yours that is working so that I can clear some bug
> possibilities out?
>
> Thanks.
>
>
>
> Quoting ikapoura@csd.auth.gr:
>
>> Ok I will check again. I just found it strange to throw that kind of error
>> without changing anything.
>>
>>
>>
>> Quoting "Edward J. Yoon" <edwardyoon@apache.org>:
>>
>>> Please see
>>> http://svn.apache.org/repos/asf/hama/trunk/examples/src/main/java/org/apache/hama/examples/PageRank.java
>>> and check your program whether it is implemented correctly or not.
>>>
>>> I guess, it's not a problem of aggregator. It's a bug in your program.
>>>
>>> On Mon, Feb 24, 2014 at 6:50 PM,  <ikapoura@csd.auth.gr> wrote:
>>>>
>>>> This is the code. What changed and suddenly it throws the error?
>>>>
>>>>
>>>>
>>>> package corever2;
>>>> import java.io.DataInput;
>>>> import java.io.DataOutput;
>>>> import java.io.IOException;
>>>> import org.apache.hadoop.io.DoubleWritable;
>>>> import org.apache.hadoop.io.IntWritable;
>>>> import org.apache.hadoop.io.Text;
>>>> import org.apache.hadoop.io.Writable;
>>>>
>>>> public class MessageCore implements Writable {
>>>>
>>>>    private DoubleWritable type;
>>>>    private Text nodeID;
>>>>    private IntWritable degree;
>>>>    private IntWritable maxDegree;
>>>>    private IntWritable nodeCounter;
>>>>
>>>>    public MessageCore() {
>>>>        type = new DoubleWritable();
>>>>        nodeID = new Text();
>>>>        degree = new IntWritable();
>>>>        maxDegree = new IntWritable();
>>>>        nodeCounter = new IntWritable();
>>>>    }
>>>>
>>>>    public MessageCore(DoubleWritable type, Text nodeID, IntWritable
>>>> degree,
>>>> IntWritable maxDegree, IntWritable nodeCounter) {
>>>>        this.type = type;
>>>>        this.nodeID = nodeID;
>>>>        this.degree = degree;
>>>>        this.maxDegree = maxDegree;
>>>>        this.nodeCounter = nodeCounter;
>>>>    }
>>>>
>>>>    public MessageCore(DoubleWritable type, Text nodeID) {
>>>>        this.type = type;
>>>>        this.nodeID = nodeID;
>>>>        degree = new IntWritable();
>>>>        maxDegree = new IntWritable();
>>>>        nodeCounter = new IntWritable();
>>>>    }
>>>>
>>>>    public void setNodeCounter(IntWritable nodeCounter) {
>>>>        this.nodeCounter = nodeCounter;
>>>>    }
>>>>
>>>>    public void setMaxDegree(IntWritable maxDegree) {
>>>>        this.maxDegree = maxDegree;
>>>>    }
>>>>
>>>>    public void setType(DoubleWritable type) {
>>>>        this.type = type;
>>>>    }
>>>>
>>>>    public void setNodeID(Text nodeID) {
>>>>        this.nodeID = nodeID;
>>>>    }
>>>>
>>>>    public DoubleWritable getType() {
>>>>        return type;
>>>>    }
>>>>
>>>>    public Text getNodeID() {
>>>>        return nodeID;
>>>>    }
>>>>
>>>>    public IntWritable getDegree() {
>>>>        return degree;
>>>>    }
>>>>
>>>>    public IntWritable getMaxDegree() {
>>>>        return maxDegree;
>>>>    }
>>>>
>>>>    public IntWritable getNodeCounter() {
>>>>        return nodeCounter;
>>>>    }
>>>>
>>>>    public void setDegree(IntWritable degree) {
>>>>        this.degree = degree;
>>>>    }
>>>>
>>>>    @Override
>>>>    public String toString() {
>>>>        return nodeID + "\t" + degree;
>>>>    }
>>>>
>>>>    @Override
>>>>    public void write(DataOutput d) throws IOException {
>>>>        type.write(d);
>>>>        nodeID.write(d);
>>>>        degree.write(d);
>>>>        maxDegree.write(d);
>>>>        nodeCounter.write(d);
>>>>    }
>>>>
>>>>    @Override
>>>>    public void readFields(DataInput di) throws  IOException {
>>>>        type.readFields(di);
>>>>        nodeID.readFields(di);
>>>>        degree.readFields(di);
>>>>        maxDegree.readFields(di);
>>>>        nodeCounter.readFields(di);
>>>>
>>>>    }
>>>> }
>>>>
>>>>
>>>>
>>>> Quoting "Edward J. Yoon" <edwardyoon@apache.org>:
>>>>
>>>>> Please attach your MessageCore.
>>>>>
>>>>> On Sun, Feb 23, 2014 at 11:37 PM,  <ikapoura@csd.auth.gr> wrote:
>>>>>>
>>>>>>
>>>>>> Hello there,
>>>>>>
>>>>>> i downloaded hama-trunk a few days ago and till now I am trying to
>>>>>> resolve
>>>>>> the following problem that came up:
>>>>>>
>>>>>> My program runs without errors without aggregators. When I set the
>>>>>> aggregator class to be used, I have the following error (which is
like
>>>>>> "cant
>>>>>> find the message class" when the java file exists in the package!):
>>>>>>
>>>>>> corever2 is the package and MessageCore is the name of the class.
>>>>>>
>>>>>> java.io.IOException: can't find class: corever2.MessageCore because
>>>>>> corever2.MessageCore
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> org.apache.hadoop.io.AbstractMapWritable.readFields(AbstractMapWritable.java:204)
>>>>>>        at
>>>>>> org.apache.hadoop.io.MapWritable.readFields(MapWritable.java:146)
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> org.apache.hama.graph.GraphJobMessage.readFields(GraphJobMessage.java:129)
>>>>>>        at
>>>>>> org.apache.hama.bsp.BSPMessageBundle$1.next(BSPMessageBundle.java:108)
>>>>>>        at
>>>>>> org.apache.hama.bsp.BSPMessageBundle$1.next(BSPMessageBundle.java:78)
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> org.apache.hama.bsp.LocalBSPRunner$LocalMessageManager.transfer(LocalBSPRunner.java:356)
>>>>>>        at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:381)
>>>>>>        at
>>>>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:133)
>>>>>>        at
>>>>>>
>>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:258)
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:288)
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:212)
>>>>>>        at
>>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>>        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>>        at
>>>>>>
>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>>>>>        at
>>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>>        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>>>>>        at java.lang.Thread.run(Thread.java:662)
>>>>>> java.io.EOFException
>>>>>>        at java.io.DataInputStream.readFully(DataInputStream.java:180)
>>>>>>        at org.apache.hadoop.io.Text.readFields(Text.java:265)
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> org.apache.hama.graph.GraphJobMessage.readFields(GraphJobMessage.java:136)
>>>>>>        at
>>>>>> org.apache.hama.bsp.BSPMessageBundle$1.next(BSPMessageBundle.java:108)
>>>>>>        at
>>>>>> org.apache.hama.bsp.BSPMessageBundle$1.next(BSPMessageBundle.java:78)
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> org.apache.hama.bsp.LocalBSPRunner$LocalMessageManager.transfer(LocalBSPRunner.java:356)
>>>>>>        at org.apache.hama.bsp.BSPPeerImpl.sync(BSPPeerImpl.java:381)
>>>>>>        at
>>>>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:133)
>>>>>>        at
>>>>>>
>>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:258)
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:288)
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:212)
>>>>>>        at
>>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>>        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>>        at
>>>>>>
>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>>>>>        at
>>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>>        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>>>>>        at java.lang.Thread.run(Thread.java:662)
>>>>>> 14/02/23 18:30:23 ERROR bsp.LocalBSPRunner: Exception during BSP
>>>>>> execution!
>>>>>> java.lang.UnsupportedOperationException: Unknown message type:
>>>>>> GraphJobMessage [flag=0, map=null, vertexId=, vertexValue=null]
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> org.apache.hama.graph.GraphJobRunner.parseMessages(GraphJobRunner.java:590)
>>>>>>        at
>>>>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:136)
>>>>>>        at
>>>>>>
>>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:258)
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:288)
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:212)
>>>>>>        at
>>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>>        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>>        at
>>>>>>
>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>>>>>        at
>>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>>        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>>>>>        at java.lang.Thread.run(Thread.java:662)
>>>>>> 14/02/23 18:30:23 ERROR bsp.LocalBSPRunner: Exception during BSP
>>>>>> execution!
>>>>>> java.util.concurrent.ExecutionException:
>>>>>> java.lang.UnsupportedOperationException: Unknown message type:
>>>>>> GraphJobMessage [flag=0, map=null, vertexId=, vertexValue=null]
>>>>>>        at
>>>>>> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>>>>>>        at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> org.apache.hama.bsp.LocalBSPRunner$ThreadObserver.run(LocalBSPRunner.java:315)
>>>>>>        at java.lang.Thread.run(Thread.java:662)
>>>>>> Caused by: java.lang.UnsupportedOperationException: Unknown message
>>>>>> type:
>>>>>> GraphJobMessage [flag=0, map=null, vertexId=, vertexValue=null]
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> org.apache.hama.graph.GraphJobRunner.parseMessages(GraphJobRunner.java:590)
>>>>>>        at
>>>>>> org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:136)
>>>>>>        at
>>>>>>
>>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.run(LocalBSPRunner.java:258)
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:288)
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> org.apache.hama.bsp.LocalBSPRunner$BSPRunner.call(LocalBSPRunner.java:212)
>>>>>>        at
>>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>>        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>>        at
>>>>>>
>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
>>>>>>        at
>>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>>>        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>>>>>        at
>>>>>>
>>>>>>
>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>>>>>        ... 1 more
>>>>>> 14/02/23 18:30:25 INFO bsp.BSPJobClient: Current supersteps number:
1
>>>>>> 14/02/23 18:30:25 INFO bsp.BSPJobClient: Job failed.
>>>>>>
>>>>>>
>>>>>> The only thing I managed to figure is that this comes up when the
>>>>>> aggregators are enabled.
>>>>>>
>>>>>> Thank you.
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Edward J. Yoon (@eddieyoon)
>>>>> Chief Executive Officer
>>>>> DataSayer, Inc.
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Edward J. Yoon (@eddieyoon)
>>> Chief Executive Officer
>>> DataSayer, Inc.
>
>
>
>



-- 
Edward J. Yoon (@eddieyoon)
Chief Executive Officer
DataSayer, Inc.

Mime
View raw message