Hi,

Why does ContinuousExecution.commit serialize an offset to JSON format just before deserializing it back (from JSON to an offset)? [1]

val offset =
          sources(0).deserializeOffset(offsetLog.get(epoch).get.offsets(0).get.json)
 
[1] https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/continuous/ContinuousExecution.scala#L341

Pozdrawiam,
Jacek Laskowski
----
The Internals of Spark SQL https://bit.ly/spark-sql-internals
The Internals of Spark Structured Streaming https://bit.ly/spark-structured-streaming
The Internals of Apache Kafka https://bit.ly/apache-kafka-internals