spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cody Koeninger <c...@koeninger.org>
Subject Re: Get Offset when using Spark Streaming + Kafka
Date Sun, 06 Mar 2016 16:51:14 GMT
Have you read the materials linked from
https://github.com/koeninger/kafka-exactly-once

On Sun, Mar 6, 2016 at 8:39 AM, Zhun Shen <shenzhunallen@gmail.com> wrote:
> Hi,
>
> I use KafkaUtils.createDirectStream to consumer data from Kafka, but I found
> that Zookeeper-based Kafka monitoring tools could not show progress of the
> streaming application because createDirectStream save the offset in
> checkpoints(http://spark.apache.org/docs/latest/streaming-kafka-integration.html),
> but I could not understand the code which shows how to update Zookeeper by
> myself.
>
> Can someone please help me to solve it?
>
> Thanks.
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message