spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacek Laskowski <ja...@japila.pl>
Subject Re: Data source v2 streaming sinks does not support Update mode
Date Tue, 12 Jan 2021 12:30:49 GMT
Hi,

Can you post the whole message? I'm trying to find what might be causing
it. A small reproducible example would be of help too. Thank you.

Pozdrawiam,
Jacek Laskowski
----
https://about.me/JacekLaskowski
"The Internals Of" Online Books <https://books.japila.pl/>
Follow me on https://twitter.com/jaceklaskowski

<https://twitter.com/jaceklaskowski>


On Tue, Jan 12, 2021 at 6:35 AM Eric Beabes <mailinglists19@gmail.com>
wrote:

> Trying to port my Spark 2.4 based (Structured) streaming application to
> Spark 3.0. I compiled it using the dependency given below:
>
> <dependency>
>     <groupId>org.apache.spark</groupId>
>     <artifactId>spark-sql-kafka-0-10_${scala.binary.version}</artifactId>
>     <version>3.1.0</version>
> </dependency>
>
>
> Every time I run it under Spark 3.0, I get this message: *Data source v2
> streaming sinks does not support Update mode*
>
> I am using '*mapGroupsWithState*' so as per this link (
> https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#output-modes),
> the only supported Output mode is "*Update*".
>
> My Sink is a Kafka topic so I am using this:
>
> .writeStream
> .format("kafka")
>
>
> What am I missing?
>
>
>

Mime
View raw message