spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Ash <and...@andrewash.com>
Subject Re: Serializable incompatible with Externalizable error
Date Tue, 03 Dec 2013 06:45:50 GMT
At least from
http://stackoverflow.com/questions/817853/what-is-the-difference-between-serializable-and-externalizable-in-javait
looks like Externalizable is roughly an old-java version of
Serializable.  Does that class implement both interfaces?  Can you take
away the Externalizable interface if it's being used?


On Mon, Dec 2, 2013 at 7:15 PM, Matt Cheah <mcheah@palantir.com> wrote:

>  Hi everyone,
>
>  I'm running into a case where I'm creating a Java RDD of an
> Externalizable class, and getting this stack trace:
>
>  java.io.InvalidClassException (java.io.InvalidClassException:
> com.palantir.finance.datatable.server.spark.WritableDataRow; Serializable
> incompatible with Externalizable)
> java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:634)
> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1622)
> java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
> java.io.ObjectInputStream.readClass(ObjectInputStream.java:1483)<some
> other Java stuff>
> java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
> org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:39)
> org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:61)
> org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:153)
> I'm running on a spark cluster generated by the EC2 Scripts. This doesn't
> happen if I'm running things with local[N]. Any ideas?
> Thanks,
> -Matt Cheah
>

Mime
View raw message