kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Su She <suhsheka...@gmail.com>
Subject Re: Can I run Kafka Producer in BG and then run a program that outputs to the console?
Date Wed, 21 Jan 2015 00:09:34 GMT
Ahh okay got it. Thank you Magnus!


On Tue, Jan 20, 2015 at 1:55 PM, Magnus Edenhill <magnus@edenhill.se> wrote:

> If you pipe your java program to kafkacat then the java program's stdout
> (output)
> will be directed to kafkacat's stdin (input).
>
>   java program | kafkacat
>
> 2015-01-20 22:48 GMT+01:00 Su She <suhshekar52@gmail.com>:
>
> > Thanks Magnus,
> >
> > One clarification...If I have a java program outputting messages, that is
> > stdout, can the producer use stdout or only stdin?
> >
> > Thanks,
> >
> > Su
> >
> > On Tue, Jan 20, 2015 at 1:10 PM, Magnus Edenhill <magnus@edenhill.se>
> > wrote:
> >
> > > Hi Su,
> > >
> > > It is as simple as piping to it:
> > >  some_program_outputting_the_stuff | kafkacat -b mybroker -t mytopic
> > >
> > > That will produce one kafka message to topic 'mytopic' for each line of
> > > input.
> > > See kafkacat -h for the full range of options (e.g., delimiter,
> specific
> > > partition, etc).
> > >
> > > Hope that helps,
> > > Magnus
> > >
> > > 2015-01-20 21:58 GMT+01:00 Su She <suhshekar52@gmail.com>:
> > >
> > > > Ahh yea that is what I was looking for, thank you! I only need it on
> > the
> > > > producer side, are there any other options for that?
> > > >
> > > > Thanks!
> > > >
> > > > On Tue, Jan 20, 2015 at 12:46 PM, Joe Stein <joe.stein@stealth.ly>
> > > wrote:
> > > >
> > > > > This is a stdin/stdout producer/consumer that works great for
> (what I
> > > > > think) you are trying to-do https://github.com/edenhill/kafkacat
> > > > >
> > > > > /*******************************************
> > > > >  Joe Stein
> > > > >  Founder, Principal Consultant
> > > > >  Big Data Open Source Security LLC
> > > > >  http://www.stealth.ly
> > > > >  Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop
> >
> > > > > ********************************************/
> > > > >
> > > > > On Tue, Jan 20, 2015 at 3:44 PM, Su She <suhshekar52@gmail.com>
> > wrote:
> > > > >
> > > > > > Hello Everyone,
> > > > > >
> > > > > > Sorry for asking multiple questions, but I am currently trying
> > > another
> > > > > > approach to run a kafka producer.
> > > > > >
> > > > > > 1) I started the kafka console producer as mentioned here in
the
> > > > > background
> > > > > > (just added a & to the end of the producer script) :
> > > > > > http://kafka.apache.org/documentation.html#introduction
> > > > > >
> > > > > > 2) I then ran a Java script that publishes messages to the
> console,
> > > > like
> > > > > > Hello World, etc, but the messages did not get published (I
know
> > the
> > > > > > connection was set up as if I manually typed in messages, I
was
> > able
> > > to
> > > > > see
> > > > > > them in my consumer).
> > > > > >
> > > > > > 3) Is this set up possible?
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > -Su
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message