spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ivan Petrov <capacyt...@gmail.com>
Subject Re: sparksql 2.4.0 java.lang.NoClassDefFoundError: com/esotericsoftware/minlog/Log
Date Thu, 09 Jul 2020 18:45:45 GMT
spark/jars -> minlog-1.3.0.jar

I see that jar is there. What do I do wrong?


чт, 9 июл. 2020 г. в 20:43, Ivan Petrov <capacytron@gmail.com>:

> Hi there!
> I'm seeing this exception in Spark Driver log.
> Executor log stays empty. No exceptions, nothing.
> 8 tasks out of 402 failed with this exception.
> What is the right way to debug it?
> Thank you.
>
> java.lang.NoClassDefFoundError: com/esotericsoftware/minlog/Log
> at
> com.esotericsoftware.kryo.serializers.FieldSerializerConfig.<init>(FieldSerializerConfig.java:47)
> at com.esotericsoftware.kryo.Kryo.<init>(Kryo.java:154)
> at com.esotericsoftware.kryo.Kryo.<init>(Kryo.java:161)
> at com.twitter.chill.KryoBase.<init>(KryoBase.scala:34)
> at
> com.twitter.chill.EmptyScalaKryoInstantiator.newKryo(ScalaKryoInstantiator.scala:36)
> at
> org.apache.spark.serializer.KryoSerializer.newKryo(KryoSerializer.scala:97)
> at
> org.apache.spark.serializer.KryoSerializerInstance.borrowKryo(KryoSerializer.scala:324)
> at
> org.apache.spark.serializer.KryoSerializerInstance.<init>(KryoSerializer.scala:309)
> at
> org.apache.spark.serializer.KryoSerializer.newInstance(KryoSerializer.scala:218)
> at
> org.apache.spark.broadcast.TorrentBroadcast$.unBlockifyObject(TorrentBroadcast.scala:305)
> at
> org.apache.spark.broadcast.TorrentBroadcast$$anonfun$readBroadcastBlock$1$$anonfun$apply$2.apply(TorrentBroadcast.scala:235)
> at scala.Option.getOrElse(Option.scala:121)
> at
> org.apache.spark.broadcast.TorrentBroadcast$$anonfun$readBroadcastBlock$1.apply(TorrentBroadcast.scala:211)
> at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1371)
> at
> org.apache.spark.broadcast.TorrentBroadcast.readBroadcastBlock(TorrentBroadcast.scala:207)
> at
> org.apache.spark.broadcast.TorrentBroadcast._value$lzycompute(TorrentBroadcast.scala:66)
> at
> org.apache.spark.broadcast.TorrentBroadcast._value(TorrentBroadcast.scala:66)
> at
> org.apache.spark.broadcast.TorrentBroadcast.getValue(TorrentBroadcast.scala:96)
> at org.apache.spark.broadcast.Broadcast.value(Broadcast.scala:70)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:89)
> at
> org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
> at org.apache.spark.scheduler.Task.run(Task.scala:121)
> at
> org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
> at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1405)
> at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>

Mime
View raw message