kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vasily Sulatskov <Vasily.Sulats...@cfm.fr>
Subject Re: Ordering of messages in the same kafka streams sub-topology with multiple sinks for the same topic
Date Wed, 04 Dec 2019 14:42:38 GMT
Hi John,

Thanks for you explanation.

I guess I'll have to adapt my code to assume there's no ordering guarantee between different
"branches".

On 2019/12/03 15:59:59, "John Roesler" <v...@apache.org> wrote: 
> Hi Vasily,> 
> 
> Probably in this case, with the constraints you're providing, the first branch would
output first, but I wouldn't depend on it. Any small change in your program could mess this
up, and also any change in Streams could alter the exact execution order also. > 
> 
> The right way to think about these programs is as "data flows". You're taking a stream
of data and defining two separate branches into smaller streams, and then later on merging
those back into one stream. In general, there would be no defined ordering, just like if you
imagine doing the same thing with literal water streams. > 
> 
> If you want a guarantee about the relative ordering, You'd have to use a specific operator
that does what you want. If nothing else comes to mind, then a custom transformer or processor
that gets records from both branches, and buffers records from the second so that it can emit
the record from the first branch first would do the trick. > 
> 
> Thanks,> 
> John> 
> 
> On Tue, Dec 3, 2019, at 06:32, Vasily Sulatskov wrote:> 
> > Hello,> 
> > > 
> > I wonder if ordering of the messages is preserved by kafka streams when > 
> > the messa

Mime
View raw message