kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sachin Mittal <sjmit...@gmail.com>
Subject Re: How to merge two topics in kafka?
Date Mon, 10 Oct 2016 06:54:00 GMT
Check this example
https://github.com/apache/kafka/blob/trunk/streams/examples/src/main/java/org/apache/kafka/streams/examples/pipe/PipeDemo.java

On Mon, Oct 10, 2016 at 11:34 AM, Ratha v <vijayaratha@gmail.com> wrote:

> Hi Sachin;
> I went through the KStream/KTable Documentation. My scenario is very
> simple..I want to merge two topics( ie: Send messages available in the
> topic A -->topic B , in my case i'll be having only single message in that
> topicA)
>
> Do I need Stateful processing (KStream)?
>
> Thanks.
>
> On 10 October 2016 at 13:10, Ratha v <vijayaratha@gmail.com> wrote:
>
> > Thank you.I'll try the solution.
> > But in the highlevel consumer API, topics will be created automatically?
> ,
> > We are not using zookeeper?
> >
> > On 10 October 2016 at 12:34, Sachin Mittal <sjmittal@gmail.com> wrote:
> >
> >> You can use topicA.leftJoin (topicB).to (new-topic).
> >>
> >> You can the consume message from that new topic via second process. Note
> >> you need to create all three topics in zookeeper first.
> >>
> >> On 10 Oct 2016 5:19 a.m., "Ratha v" <vijayaratha@gmail.com> wrote:
> >>
> >> Hi all;
> >>
> >> I have two topics in the broker. Without consuming from one topic, I
> want
> >> to merge both topics, and will consume messages from the second topic.
> >>
> >> It is because, I have two processes, one process, pushes messages to
> topic
> >> A. And the second process once finished processing, it wants to merge
> both
> >> topicA and TopicB. Then another process will consume messages from the
> >> merged topic.
> >>
> >> How can I merge both topics in high level kafka APIs. I use Kafka
> >> 0.10.0.1.
> >>
> >>
> >> Thanks.
> >> --
> >> -Ratha
> >> http://vvratha.blogspot.com/
> >>
> >
> >
> >
> > --
> > -Ratha
> > http://vvratha.blogspot.com/
> >
>
>
>
> --
> -Ratha
> http://vvratha.blogspot.com/
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message