kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Bejeck <b...@confluent.io>
Subject Re: Referential transparency issue with transform()
Date Mon, 24 Sep 2018 14:25:18 GMT
Hi Stéphane,

Can you provide log files from the crash so we can see the exact failure?

Thanks,
Bill

On Mon, Sep 24, 2018 at 8:12 AM Stéphane. D. <derste@gmail.com> wrote:

> Hi,
>
> We just stumbled upon an issue with KStream.transform() where we had a
> runtime error with this code:
>
> ```
> DeduplicationTransformer<X, Y, , > transformer = new
> DeduplicationTransformer<>(...);
> stream.transform(() -> transformer, ...)
> ```
>
> The error was:
> Failed to process stream task 0_0 due to the following error:
> java.lang.IllegalStateException: This should not happen as timestamp()
> should only be called while a record is processed
>
> Whereas simply inlining the creation of the Transformer works perfectly:
>
> ```
> stream.transform(() -> new DeduplicationTransformer<>(...), ...)
> ```
>
> Is this behavior expected?
>
>
> Is that why transform() takes a lazy function, to access some global
> state when needed at execution? (and breaking referential
> transparency)
>
>
> Thanks,
>
> Stéphane
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message