spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eduardo Berrocal <eberr...@hawk.iit.edu>
Subject Re: java.lang.AbstractMethodError
Date Thu, 03 Oct 2013 21:30:23 GMT
Ok, I found the mistake!. Wow, it really come to me by inspiration, if not
I don't know how this suddenly come to me.

The problem is that I packed my application in a jar file with the old
"spark-assembly-0.8.0-incubating-hadoop1.0.4.jar" in it. So that is the
reason why workers and master have different versions of the file. I
re-packeted it and run it just fine!.

Thanks for all the help,
Eduardo.


On Thu, Oct 3, 2013 at 4:05 PM, Eduardo Berrocal <eberroca@hawk.iit.edu>wrote:

> And by .java, I mean .scala
>
>
> On Thu, Oct 3, 2013 at 4:03 PM, Eduardo Berrocal <eberroca@hawk.iit.edu>wrote:
>
>> That directory is not present on version 0.8.0 (the one I am using).
>>
>> However, there are the files for FlatMapFunction in spark 0.8.0:
>>
>> $ find ./ -name FlatMapFunction*
>>
>> ./assembly/target/scala-2.9.3/cache/assembly/global/assembly/046f8c33ceea082eba335f924f10506f884e593c_dir/org/apache/spark/api/java/function/FlatMapFunction2.class
>>
>> ./assembly/target/scala-2.9.3/cache/assembly/global/assembly/046f8c33ceea082eba335f924f10506f884e593c_dir/org/apache/spark/api/java/function/FlatMapFunction.class
>>
>> ./core/src/main/scala/org/apache/spark/api/java/function/FlatMapFunction2.scala
>>
>> ./core/src/main/scala/org/apache/spark/api/java/function/FlatMapFunction.scala
>>
>> ./core/target/scala-2.9.3/classes/org/apache/spark/api/java/function/FlatMapFunction2.class
>>
>> ./core/target/scala-2.9.3/classes/org/apache/spark/api/java/function/FlatMapFunction.class
>>
>> ./examples/target/scala-2.9.3/cache/examples/global/assembly/046f8c33ceea082eba335f924f10506f884e593c_dir/org/apache/spark/api/java/function/FlatMapFunction2.class
>>
>> ./examples/target/scala-2.9.3/cache/examples/global/assembly/046f8c33ceea082eba335f924f10506f884e593c_dir/org/apache/spark/api/java/function/FlatMapFunction.class
>>
>> All of the FlatMapFunction.class files have a modification time bigger
>> than the .java file, which I suspect means that they have been created from
>> the new version of the java file (all of them!)
>>
>> Cheers,
>> Eduardo.
>>
>>
>> On Thu, Oct 3, 2013 at 3:38 PM, Martin Weindel <martin.weindel@gmail.com>wrote:
>>
>>>  Hi Eduardo,
>>>
>>> if you are using Spark 0.7.3, I remember that I had to replace the class
>>> file additional at
>>> spark-0.7.3/core/target/scala-2.9.3/classes/spark/api/java/function/
>>>
>>> Martin
>>>
>>> Am 03.10.2013 22:35, schrieb Eduardo Berrocal:
>>>
>>>  Hi Martin,
>>>
>>>  Yes, that is what is seems. However, it is unlikely that is the case,
>>> because I have all spark classes on my home, which is mounted on NFS to all
>>> nodes. Unless there is something else I am missing...
>>>
>>>  Edu
>>>
>>>
>>> On Thu, Oct 3, 2013 at 3:29 PM, Martin Weindel <martin.weindel@gmail.com
>>> > wrote:
>>>
>>>>  Hi Eduardo,
>>>>
>>>> it seems to me that your second problem is caused by inconsistent, i.e.
>>>> different classes in master and worker JVMs.
>>>> Are you sure, that you have replaced the changed FlatMapFunction on all
>>>> worker nodes and also on master?
>>>>
>>>> Regards,
>>>> Martin
>>>>
>>>>  13/10/03 13:27:44 INFO cluster.ClusterTaskSetManager: Lost TID 0 (task
>>>> 1.0:0)
>>>> 13/10/03 13:27:44 INFO cluster.ClusterTaskSetManager: Loss was due to
>>>> java.io.InvalidClassException
>>>> java.io.InvalidClassException:
>>>> org.apache.spark.api.java.function.FlatMapFunction; local class
>>>> incompatible: stream classdesc serialVersionUID = -1748278142466443391,
>>>> local class serialVersionUID = 2220150375729402137
>>>>
>>>>
>>>>
>>>
>>>
>>
>

Mime
View raw message