spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cody Koeninger <c...@koeninger.org>
Subject Re: Spark streaming Kafka receiver WriteAheadLog question
Date Mon, 25 Apr 2016 14:23:32 GMT
If you want to refer back to Kafka based on offset ranges, why not use
createDirectStream?

On Fri, Apr 22, 2016 at 11:49 PM, Renyi Xiong <renyixiong0@gmail.com> wrote:
> Hi,
>
> Is it possible for Kafka receiver generated WriteAheadLogBackedBlockRDD to
> hold corresponded Kafka offset range so that during recovery the RDD can
> refer back to Kafka queue instead of paying the cost of write ahead log?
>
> I guess there must be a reason here. Could anyone please help me understand?
>
> Thanks,
> Renyi.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org


Mime
View raw message