spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tathagata Das <t...@databricks.com>
Subject Re: Checkpoint Dir Error in Yarn
Date Sat, 08 Aug 2015 01:17:44 GMT
Have you tried to do what its suggesting?
If you want to learn more about checkpointing, you can see the programming
guide -
http://spark.apache.org/docs/latest/streaming-programming-guide.html#checkpointing
For more in depth understanding, you can see my talk -
https://www.youtube.com/watch?v=d5UJonrruHk

On Fri, Aug 7, 2015 at 5:48 PM, Mohit Anchlia <mohitanchlia@gmail.com>
wrote:

> I am running in yarn-client mode and trying to execute network word count
> example. When I connect through nc I see the following in spark app logs:
>
> Exception in thread "main" java.lang.AssertionError: assertion failed: The
> checkpoint directory has not been set. Please use
> StreamingContext.checkpoint() or SparkContext.checkpoint() to set the
> checkpoint directory.
>         at scala.Predef$.assert(Predef.scala:179)
>         at
> org.apache.spark.streaming.dstream.DStream.validate(DStream.scala:183)
>         at
> org.apache.spark.streaming.dstream.DStream$$anonfun$validate$10.apply(DStream.scala:229)
>         at
> org.apache.spark.streaming.dstream.DStream$$anonfun$validate$10.apply(DStream.scala:229)
>         at scala.collection
>

Mime
View raw message