spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dimitris Kouzis - Loukas <>
Subject Re: Pause Spark Streaming reading or sampling streaming data
Date Thu, 06 Aug 2015 00:27:56 GMT
What driver do you use? Sounds like something you should do before the

On Thu, Aug 6, 2015 at 12:50 AM, Heath Guo <> wrote:

> Hi, I have a question about sampling Spark Streaming data, or getting part
> of the data. For every minute, I only want the data read in during the
> first 10 seconds, and discard all data in the next 50 seconds. Is there any
> way to pause reading and discard data in that period? I'm doing this to
> sample from a stream of huge amount of data, which saves processing time in
> the real-time program. Thanks!

View raw message