spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <bryan.jeff...@gmail.com>
Subject RE: How to Check Dstream is empty or not?
Date Fri, 02 Dec 2016 00:11:29 GMT
The stream is just a wrapper over batch operations. You can check if a batch is empty by simply
doing:

val isEmpty = stream.transform(rdd => rdd.isEmpty)

This will give you a stream of Boolean indicating if given batches are empty.

Bryan Jeffrey


From: rockinfeet@gmail.com
Sent: Thursday, December 1, 2016 7:28 AM
To: user@spark.apache.org
Subject: How to Check Dstream is empty or not?

I have integerated flume with spark using Flume-style Push-based Approach. 
I need to check whether Dstream is empty. Please suggest how can i do that?



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-Check-Dstream-is-empty-or-not-tp28151.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org



Mime
View raw message