spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From MathieuP <mathieu.pre...@actility.com>
Subject Re: Checkpoints not cleaned using Spark streaming + watermarking + kafka
Date Fri, 22 Sep 2017 09:44:14 GMT
The expected setting to clean these files is :
- spark.sql.streaming.minBatchesToRetain

More info on structured streaming settings :
https://github.com/jaceklaskowski/spark-structured-streaming-book/blob/master/spark-sql-streaming-properties.adoc





--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message