spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From satyajit vegesna <satyajit.apas...@gmail.com>
Subject Spark error while trying to spark.read.json()
Date Wed, 20 Dec 2017 01:42:21 GMT
Hi All,

Can anyone help me with below error,

Exception in thread "main" java.lang.AbstractMethodError
at
scala.collection.TraversableLike$class.filterNot(TraversableLike.scala:278)
at org.apache.spark.sql.types.StructType.filterNot(StructType.scala:98)
at org.apache.spark.sql.DataFrameReader.json(DataFrameReader.scala:386)
at
org.spark.jsonDF.StructStreamKafkaToDF$.getValueSchema(StructStreamKafkaToDF.scala:22)
at org.spark.jsonDF.StructStreaming$.createRowDF(StructStreaming.scala:21)
at SparkEntry$.delayedEndpoint$SparkEntry$1(SparkEntry.scala:22)
at SparkEntry$delayedInit$body.apply(SparkEntry.scala:7)
at scala.Function0$class.apply$mcV$sp(Function0.scala:40)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App$$anonfun$main$1.apply(App.scala:76)
at scala.App$$anonfun$main$1.apply(App.scala:76)
at scala.collection.immutable.List.foreach(List.scala:381)
at
scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:35)
at scala.App$class.main(App.scala:76)
at SparkEntry$.main(SparkEntry.scala:7)
at SparkEntry.main(SparkEntry.scala)

This happening, when i try to pass Dataset[String] containing jsons to
spark.read.json(Records).

Regards,
Satyajit.

Mime
View raw message