kylin-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "xiangakun (Jira)" <j...@apache.org>
Subject [jira] [Commented] (KYLIN-4102) java.lang.VerifyError: class com.google.common.collect.EmptyImmutableList overrides final method toArray.([Ljava/lang/Object;)[Ljava/lang/Object;
Date Tue, 07 Apr 2020 02:58:00 GMT

    [ https://issues.apache.org/jira/browse/KYLIN-4102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17076843#comment-17076843
] 

xiangakun commented on KYLIN-4102:
----------------------------------

In my case,the guava jar is assigned by the config spark.executor.extraClassPath in our
production env,and it can be overrided by the config :kylin.engine.spark-conf.spark.executor.extraClassPath
in kylin.properties file,you can also search “guava” in spark application web UI —>
Environment tab. Hope it will help!



> java.lang.VerifyError: class com.google.common.collect.EmptyImmutableList overrides final
method toArray.([Ljava/lang/Object;)[Ljava/lang/Object;
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: KYLIN-4102
>                 URL: https://issues.apache.org/jira/browse/KYLIN-4102
>             Project: Kylin
>          Issue Type: Bug
>    Affects Versions: v2.6.2
>         Environment: Kylin2.6.2 ,  hadoop-2.6.0-cdh5.13.1.1, hbase-1.2.0-cdh5.10.1, 
hive-2.3.4, spark-2.3.2
>            Reporter: xiangakun
>            Priority: Major
>
> Hi,
>  
> When I used spark engine to build cube, and met below error in step #7, it seems the
guava jar with confliction, is it an issue?
>  
> 19/07/19 10:13:55 INFO DAGScheduler: ShuffleMapStage 0 (mapToPair at SparkCubingByLayer.java:169)
failed in 9.119 s due to Job aborted due to stage failure: Task 6 in stage 0.0 failed 4 times,
most recent failure: Lost task 6.3 in stage 0.0 (TID 18, sd-hadoop-datanode-5
> 1-131.idc.vip.com, executor 2): java.lang.VerifyError: class com.google.common.collect.EmptyImmutableList
overrides final method toArray.([Ljava/lang/Object;)[Ljava/lang/Object; at java.lang.ClassLoader.defineClass1(Native
Method)
>  at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
>  at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>  at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
>  at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
>  at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
>  at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
>  at java.security.AccessController.doPrivileged(Native Method)
>  at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
>  at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
>  at java.lang.ClassLoader.loadClass(ClassLoader.java:411)
>  at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>  at java.lang.Class.forName0(Native Method)
>  at java.lang.Class.forName(Class.java:264)
>  at org.apache.kylin.engine.spark.KylinKryoRegistrator.addClassQuitely(KylinKryoRegistrator.java:312)
>  at org.apache.kylin.engine.spark.KylinKryoRegistrator.registerClasses(KylinKryoRegistrator.java:109)
>  at org.apache.spark.serializer.KryoSerializer$$anonfun$newKryo$6.apply(KryoSerializer.scala:136)
>  at org.apache.spark.serializer.KryoSerializer$$anonfun$newKryo$6.apply(KryoSerializer.scala:136)
>  at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
>  at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
>  at org.apache.spark.serializer.KryoSerializer.newKryo(KryoSerializer.scala:136)
>  at org.apache.spark.serializer.KryoSerializerInstance.borrowKryo(KryoSerializer.scala:324)
>  at org.apache.spark.serializer.KryoSerializerInstance.<init>(KryoSerializer.scala:309)
>  at org.apache.spark.serializer.KryoSerializer.newInstance(KryoSerializer.scala:218)
>  at org.apache.spark.broadcast.TorrentBroadcast$.unBlockifyObject(TorrentBroadcast.scala:305)
>  at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$readBroadcastBlock$1$$anonfun$apply$2.apply(TorrentBroadcast.scala:235)
>  at scala.Option.getOrElse(Option.scala:121)
>  at org.apache.spark.broadcast.TorrentBroadcast$$anonfun$readBroadcastBlock$1.apply(TorrentBroadcast.scala:211)
>  at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1347)
>  at org.apache.spark.broadcast.TorrentBroadcast.readBroadcastBlock(TorrentBroadcast.scala:207)
>  at org.apache.spark.broadcast.TorrentBroadcast._value$lzycompute(TorrentBroadcast.scala:66)
>  at org.apache.spark.broadcast.TorrentBroadcast._value(TorrentBroadcast.scala:66)
>  at org.apache.spark.broadcast.TorrentBroadcast.getValue(TorrentBroadcast.scala:96)
>  at org.apache.spark.broadcast.Broadcast.value(Broadcast.scala:70)
>  at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:86)
>  at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
>  at org.apache.spark.scheduler.Task.run(Task.scala:109)
>  at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
>  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>  at java.lang.Thread.run(Thread.java:745)
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message