spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rabin Banerjee <dev.rabin.baner...@gmail.com>
Subject Re: Latest 200 messages per topic
Date Sat, 16 Jul 2016 15:43:14 GMT
Just to add ,

  I want to read the MAX_OFFSET of a topic , then read MAX_OFFSET-200 ,
every time .

Also I want to know , If I want to fetch a specific offset range for Batch
processing, is there any option for doing that ?



On Sat, Jul 16, 2016 at 9:08 PM, Rabin Banerjee <
dev.rabin.banerjee@gmail.com> wrote:

> HI All,
>
>    I have 1000 kafka topics each storing messages for different devices .
> I want to use the direct approach for connecting kafka from Spark , in
> which I am only interested in latest 200 messages in the Kafka .
>
> How do I do that ?
>
> Thanks.
>

Mime
View raw message