kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Damian Guy <damian....@gmail.com>
Subject Re: No referential transparency with transform() ?
Date Mon, 24 Sep 2018 15:17:26 GMT
The return value from the `TransformSupplier` should always be a `new
YourTransformer(..)` as there will be one for each task and they are
potentially processed on multiple threads.

On Mon, 24 Sep 2018 at 16:07 St├ęphane. D. <derste@gmail.com> wrote:

> Hi,
>
> We just stumbled upon an issue with KStream.transform() where we had a
> runtime error with this code:
>
> ```
> DeduplicationTransformer<X, Y, , > transformer = new
> DeduplicationTransformer<>(...);
> stream.transform(() -> transformer, ...)
> ```
>
> The error is:
> Failed to process stream task 0_0 due to the following error:
> java.lang.IllegalStateException: This should not happen as timestamp()
> should only be called while a record is processed
>
> Whereas simply inlining the creation of the Transformer works:
>
> ```
> stream.transform(() -> new DeduplicationTransformer<>(...), ...)
> ```
>
> Is this behavior expected?
>
>
> I guess that's why tranform() takes a wrapper, to construct it when needed?
>
> Why does this happen? Is there some kind of global reference used
> internally (only construct during the execution) ?
>
>
> Thanks,
>
> St├ęphane
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message