spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jordan Pilat <>
Subject Spark UI Streaming batch time interval does not match batch interval
Date Mon, 12 Mar 2018 21:22:25 GMT

I am running a streaming app on Spark 2.1.2.  The batch interval is set to 5000ms, and when
I go to the "Streaming" tab in the Spark UI, it correctly reports a 5 second batch interval,
but the list of batches below only shows one batch every two minutes (IE the batch time for
each batch is two minutes after the prior batch).

Keep in mind, I am not referring to delay or processing time -- only when each batch starts.
 What could account for this discrepancy?

To unsubscribe e-mail:

View raw message