storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniccan VP <danic...@iqsystech.com>
Subject Offset out of range error in Kafka Spout
Date Fri, 14 Oct 2016 10:32:38 GMT
Hi,

I am currently working on a project which involves Storm and Kafka interaction. I am using
Kafka Spout which would consume the messages from the Kafka Brokers followed by the processing
using the ATDM Bolt.

Now, the issue that I am facing is, sometimes when I run my Storm topology after a delay,
I get a error, "Got fetch request with offset out of range" and this run in a indefinite loop.
This is probably because the Log Retention period in Kafka has expired. But, is there a way
to change the offset dynamically to the next available offset if this error occurs.

Thanks and Regards,
Daniccan VP | Junior Software Engineer
Email : daniccan@iqsystech.com
*******************************************************************************************************************************************************************
This email and any files transmitted with it are confidential and intended solely for the
use of the individual or entity to whom they have been addressed. If you are not the intended
recipient, you are notified that disclosing, copying, distributing or taking any action in
reliance on the contents of this information is strictly prohibited. Please notify the sender
immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from
your system.

Mime
View raw message