Thanks TD.

On Mon, Aug 7, 2017 at 8:59 PM, Tathagata Das <tathagata.das1565@gmail.com> wrote:
I dont think there is any easier way.

On Mon, Aug 7, 2017 at 7:32 PM, shyla deshpande <deshpandeshyla@gmail.com> wrote:
Thanks TD for the response. I forgot to mention that I am not using structured streaming. 

I was looking into KafkaUtils.createRDD, and looks like I need to get the earliest and the latest offset for each partition to build the Array(offsetRange). I wanted to know if there was a easier way. 

1 reason why we are hesitating to use structured streaming is because I need to persist the data in Cassandra database which I believe is not production ready.


On Mon, Aug 7, 2017 at 6:11 PM, Tathagata Das <tathagata.das1565@gmail.com> wrote:

On Mon, Aug 7, 2017 at 6:03 PM, shyla deshpande <deshpandeshyla@gmail.com> wrote:
Hi all,

What is the easiest way to read all the data from kafka in a batch program for a given topic? 
I have 10 kafka partitions, but the data is not much. I would like to read  from the earliest from all the partitions for a topic.

I appreciate any help. Thanks