spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Starks <suse...@protonmail.com.INVALID>
Subject Data source jdbc does not support streamed reading
Date Wed, 08 Aug 2018 16:23:54 GMT
Now my spark job can perform sql operations against database table. Next I want to combine
 that with streaming context, so switching to readStream() function. But after job submission,
spark throws

    Exception in thread "main" java.lang.UnsupportedOperationException: Data source jdbc does
not support streamed reading

That looks like sparkSession.readSteam.format("jdbc")... jdbc doesn't support streaming

    val sparkSession = SparkSession.builder().appName("my-test").getOrCreate()
    import session.implicits._
    val df = sparkSession.readStream.format("jdbc")...load()
    // other operations against df

Checking the example - https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/sql/streaming/StructuredSessionization.scala

Also searching on the internet, I don't see any examples that close to my need. Any pointers
or docs that may talk about this or code snippet that may illustrate such purpose?

Thanks
Mime
View raw message