spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kineret M <kiner...@gmail.com>
Subject Why planInputPartitions is called multiple times in a micro-batch?
Date Mon, 12 Jul 2021 10:33:36 GMT
Hi,

I'm developing a new Spark connector using data source v2 API (spark 3.1.1).
I noticed that the planInputPartitions method (in MicroBatchStream) is
called twice every micro-batch.

What the motivation/reason is?

Thanks,
Kineret

Mime
View raw message