spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergey Belousov <sergey.belou...@gmail.com>
Subject Spark Cluster vs Spark on YARN jar loading
Date Wed, 04 Feb 2015 21:08:30 GMT
Hi All

We have our farjar that using asynchbase throwing following exception.

ERROR [Executor task launch worker-2-EventThread:ClientCnxn$EventThread@610]
- Caught unexpected throwable
java.lang.IllegalAccessError: class
com.google.protobuf.ZeroCopyLiteralByteString cannot access its superclass
com.google.protobuf.LiteralByteString
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(Unknown Source)
at java.security.SecureClassLoader.defineClass(Unknown Source)
at java.net.URLClassLoader.defineClass(Unknown Source)
at java.net.URLClassLoader.access$100(Unknown Source)
at java.net.URLClassLoader$1.run(Unknown Source)
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at org.hbase.async.Bytes.wrap(Bytes.java:287)
at org.hbase.async.RegionClient.<clinit>(RegionClient.java:580)
at
org.hbase.async.HBaseClient$RegionClientPipeline.init(HBaseClient.java:2655)
at org.hbase.async.HBaseClient.newClient(HBaseClient.java:2604)
at org.hbase.async.HBaseClient.access$2700(HBaseClient.java:179)
at
org.hbase.async.HBaseClient$ZKClient$ZKCallback.handleMetaZnode(HBaseClient.java:3301)
at
org.hbase.async.HBaseClient$ZKClient$ZKCallback.processResult(HBaseClient.java:3157)
at
org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:558)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)


*But when we try it with Spark on Yarn we do not have that problem. *
Can someone provide details as of what are differences that made that
particular problem go away when we use Spark on YARN?

I have some guess but I would really appreciate some infor from people who
has more than just guess.

Thank you
S

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message