spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saisai Shao <sai.sai.s...@gmail.com>
Subject Re: Spark 1.4: Python API for getting Kafka offsets in direct mode?
Date Fri, 12 Jun 2015 05:02:47 GMT
Hi,

What is your meaning of getting the offsets from the RDD, from my
understanding, the offsetRange is a parameter you offered to KafkaRDD, why
do you still want to get the one previous you set into?

Thanks
Jerry

2015-06-12 12:36 GMT+08:00 Amit Ramesh <amit@yelp.com>:

>
> Congratulations on the release of 1.4!
>
> I have been trying out the direct Kafka support in python but haven't been
> able to figure out how to get the offsets from the RDD. Looks like the
> documentation is yet to be updated to include Python examples (
> https://spark.apache.org/docs/latest/streaming-kafka-integration.html). I
> am specifically looking for the equivalent of
> https://spark.apache.org/docs/latest/streaming-kafka-integration.html#tab_scala_2.
> I tried digging through the python code but could not find anything
> related. Any pointers would be greatly appreciated.
>
> Thanks!
> Amit
>
>

Mime
View raw message