flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Smirnov <alexander.smirn...@gmail.com>
Subject Re: Tracking deserialization errors
Date Sun, 08 Apr 2018 15:53:35 GMT
I have the same question. In case of kafka source, it would be good to know
topic name and offset of the corrupted message for further investigation.
Looks like the only option is to write messages into a log file

On Fri, Apr 6, 2018 at 9:12 PM Elias Levy <fearsome.lucidity@gmail.com>

> I was wondering how are folks tracking deserialization errors.
> The AbstractDeserializationSchema interface provides no mechanism for the
> deserializer to instantiate a metric counter, and "deserialize" must return
> a null instead of raising an exception in case of error if you want your
> job to continue functioning during a deserialization error.  But that means
> such errors are invisible.
> Thoughts?

View raw message