spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <>
Subject RE: How to Check Dstream is empty or not?
Date Fri, 02 Dec 2016 00:11:29 GMT
The stream is just a wrapper over batch operations. You can check if a batch is empty by simply

val isEmpty = stream.transform(rdd => rdd.isEmpty)

This will give you a stream of Boolean indicating if given batches are empty.

Bryan Jeffrey

Sent: Thursday, December 1, 2016 7:28 AM
Subject: How to Check Dstream is empty or not?

I have integerated flume with spark using Flume-style Push-based Approach. 
I need to check whether Dstream is empty. Please suggest how can i do that?

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe e-mail:

View raw message