spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Srikanth <>
Subject Reset auto.offset.reset in Kafka 0.10 integ
Date Fri, 02 Sep 2016 19:57:33 GMT

Upon restarting my Spark Streaming app it is failing with error

Exception in thread "main" org.apache.spark.SparkException: Job aborted due
to stage failure: Task 0 in stage 1.0 failed 1 times, most recent failure:
Lost task 0.0 in stage 1.0 (TID 6, localhost):
org.apache.kafka.clients.consumer.OffsetOutOfRangeException: Offsets out of
range with no configured reset policy for partitions: {mt-event-2=1710706}

It is correct that the last read offset was deleted by kafka due to
retention period expiry.
I've set auto.offset.reset in my app but it is getting reset here

How to force it to restart in this case (fully aware of potential data


View raw message