spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saisai Shao <>
Subject Re: Spark 1.4: Python API for getting Kafka offsets in direct mode?
Date Fri, 12 Jun 2015 05:02:47 GMT

What is your meaning of getting the offsets from the RDD, from my
understanding, the offsetRange is a parameter you offered to KafkaRDD, why
do you still want to get the one previous you set into?


2015-06-12 12:36 GMT+08:00 Amit Ramesh <>:

> Congratulations on the release of 1.4!
> I have been trying out the direct Kafka support in python but haven't been
> able to figure out how to get the offsets from the RDD. Looks like the
> documentation is yet to be updated to include Python examples (
> I
> am specifically looking for the equivalent of
> I tried digging through the python code but could not find anything
> related. Any pointers would be greatly appreciated.
> Thanks!
> Amit

View raw message