spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Beabes <mailinglist...@gmail.com>
Subject Data source v2 streaming sinks does not support Update mode
Date Tue, 12 Jan 2021 05:35:20 GMT
Trying to port my Spark 2.4 based (Structured) streaming application to
Spark 3.0. I compiled it using the dependency given below:

<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-sql-kafka-0-10_${scala.binary.version}</artifactId>
    <version>3.1.0</version>
</dependency>


Every time I run it under Spark 3.0, I get this message: *Data source v2
streaming sinks does not support Update mode*

I am using '*mapGroupsWithState*' so as per this link (
https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#output-modes),
the only supported Output mode is "*Update*".

My Sink is a Kafka topic so I am using this:

.writeStream
.format("kafka")


What am I missing?

Mime
View raw message