kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ezequiel Puig <essep...@gmail.com>
Subject Re: Deserialization Exception when performing state store operations
Date Tue, 21 Apr 2020 05:38:10 GMT
Hello Carl,

I think that the problem you are facing is a "poison pill"

You can take a look at this video for approches on different solutions :
https://www.confluent.io/kafka-summit-san-francisco-2019/streaming-apps-and-poison-pills-handle-the-unexpected-with-kafka-streams/

Best regards,

Ezequiel PUIG

Le mar. 21 avr. 2020 à 04:59, Carl Graving <cgraving@gmail.com> a écrit :

> I have a state store being built from a stream with a custom set of value
> serdes. In the stream processing I am able to handle errors appropriately
> with the exception handler, but if I attempt to do a state store getAll and
> use the iterator.hasNext() or next() or peek methods, any exceptions in the
> deserialization are thrown. This is proving to be difficult to iterate over
> the items in the state store and skip bad entries or remove them. I can see
> different ways this could happen, such as avro schema removed from schema
> registry (and cache) for an item in the state store. Am I missing an easy
> way to deal with deserialization exceptions stemming from state store
> iterator or gets? I will keep playing around, but it was hard to find a way
> to reliably use the iterator (getAll) when hasNext, next, peekNext, etc all
> throw exceptions.
>
> Thanks for any help or pointers on how to properly handle exceptions in
> this case.
>
> Carl
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message