spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sadaf <>
Subject Checkpoint issue in spark streaming
Date Tue, 28 Jul 2015 13:59:26 GMT
Hi all.

I am writing a twitter connector using spark streaming. i have written the
following code to maintain checkpoint.

{ managingContext() })
def managingContext():StreamingContext =
      //making spark context
      val conf = new
       val ssc = new StreamingContext(conf, Seconds(1))
       val sqlContext = new
       import sqlContext.implicits._ 

but it gives the following error:

java.lang.IllegalArgumentException: requirement failed: WindowedDStream has
been marked for checkpointing but the storage level has not been set to
enable persisting. Please use DStream.persist() to set the storage level to
use memory for better checkpointing performance.

I have also mentioned the storage level through the following code.

Can anyone help me in this regard? 
Thanks :) 

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message