spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From M Singh <mans2si...@yahoo.com.INVALID>
Subject Apache Spark - Spark Structured Streaming - Watermark usage
Date Fri, 26 Jan 2018 18:14:16 GMT
Hi:
I am trying to filter out records which are lagging behind (based on event time) by a certain
amount of time.  
Is the watermark api applicable to this scenario (ie, filtering lagging records) or it is
only applicable with aggregation ?  I could not get a clear understanding from the documentation
which only refers to it's usage with aggregation.

Thanks
Mans
Mime
View raw message