spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kanwaldeep <kanwal...@gmail.com>
Subject Using ProtoBuf 2.5 for messages with Spark Streaming
Date Fri, 28 Mar 2014 01:41:39 GMT
We are using Protocol Buffer 2.5 to send messages to Spark Streaming 0.9 with
Kafka stream setup. I have protocol Buffer 2.5 part of the uber jar deployed
on each of the spark worker nodes.  
The message is compiled using 2.5 but then on runtime it is being
de-serialized by 2.4.1 as I'm getting the following exception

java.lang.VerifyError (java.lang.VerifyError: class
com.snc.sinet.messages.XServerMessage$XServer overrides final method
getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;)
java.lang.ClassLoader.defineClass1(Native Method)
java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
java.lang.ClassLoader.defineClass(ClassLoader.java:615)
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)

Suggestions on how I could still use ProtoBuf 2.5. Based on the article -
https://spark-project.atlassian.net/browse/SPARK-995 we should be able to
use different version of protobuf in the application.





--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Using-ProtoBuf-2-5-for-messages-with-Spark-Streaming-tp3396.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Mime
View raw message