kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bruno Cadonna <br...@confluent.io>
Subject Re: Case of joining multiple streams/tables
Date Thu, 05 Dec 2019 08:52:26 GMT
Hi Sachin,

I do not completely understand what you mean with one single
operation. Do you mean one call of a method in the DSL or the join is
processed on one processor node?

If you mean the latter, the joins in the DSL are also not processed on
one single processor node.

If you mean the former, the DSL does not have a single method call to
join multiple streams and it does not necessarily need it to process
an n-way join more efficiently, because the DSL is just the way you
declare the join. How the join is processed depends on how the
topology is build from the DSL code. Having a DSL call specific for a
n-way join would merely result in syntactic sugar (which can also make
sense).

If you have specific requirements that are not fulfilled by the DSL
you can use the Processor API to implement your own join.

See the following StackOverflow question for more details on joins.
https://stackoverflow.com/questions/53485632/kafka-streams-implementing-joining-using-the-processor-api

Best,
Bruno

On Thu, Dec 5, 2019 at 7:08 AM Sachin Mittal <sjmittal@gmail.com> wrote:
>
> Hi,
> I have checked the documentation and what I see that we can join two
> streams or tables at a given time.
>
> I have a case where I have multiple streams which I need to join based on
> common key.
>
> As of now I am first joining two and the result of that with next and so on.
>
> Is there a way or any case implemented anywhere that joins multiple
> streams/tables in a single operation.
>
> If not then is this something that is pipelined for future releases?
> Or does something like this make sense to be part of streams functionality?
>
> Thanks
> Sachin

Mime
View raw message