spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tathagata Das <tathagata.das1...@gmail.com>
Subject Re: Spark Streaming : Could not compute split, block not found
Date Fri, 01 Aug 2014 18:56:35 GMT
Are you accessing the RDDs on raw data blocks and running independent
Spark jobs on them (that is outside DStream)? In that case this may
happen as Spark Straming will clean up the raw data based on the
DStream operations (if there is a window op of 15 mins, it will keep
the data around for 15 mins at least). So independent Spark jobs that
access old data may fail. The solution for that is using
DStream.remember() on the raw input stream to make sure the data is
kept around.

Not sure if this was the problem or not. For more info can you tell
when you are running Spark 0.9 or 1.0?



TD

On Fri, Aug 1, 2014 at 10:55 AM, Kanwaldeep <kanwal239@gmail.com> wrote:
> We are using Sparks 1.0 for Spark Streaming on Spark Standalone cluster and
> seeing the following error.
>
>         Job aborted due to stage failure: Task 3475.0:15 failed 4 times, most
> recent failure: Exception failure in TID 216394 on host
> hslave33102.sjc9.service-now.com: java.lang.Exception: Could not compute
> split, block input-0-1406869340000 not found
> org.apache.spark.rdd.BlockRDD.compute(BlockRDD.scala:51)
>
> We are using the Memory_DISK serialization option for the input streams. And
> the stream is also being persisted since we have multiple transformations
> happening on the input stream.
>
>
>     val lines = KafkaUtils.createStream[String, Array[Byte], StringDecoder,
> DefaultDecoder](ssc, kafkaParams, topicpMap,
> StorageLevel.MEMORY_AND_DISK_SER)
>
>     lines.persist(StorageLevel.MEMORY_AND_DISK_SER)
>
> We are aggregating data every 15 minutes as well as an hour. The
> spark.streaming.blockInterval=10000 so we minimize the blocks of data read.
>
> The problem started at the 15 minute interval but now I'm seeing it happen
> every hour since last night.
>
> Any suggestions?
>
> Thanks
> Kanwal
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-Could-not-compute-split-block-not-found-tp11186.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.

Mime
View raw message