kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prabhjot Bharaj <prabhbha...@gmail.com>
Subject Re: New and updated producers and consumers
Date Thu, 05 Nov 2015 18:23:02 GMT
Hi Jeff,

Thanks for your response.
On scala side, is there a Producer implementation that I could use? is the
java based KafkaProducer (org.apache.kafka.clients.producer.KafkaProducer;)
same as Producer in Producer.scala ?

Thanks,
Prabhjot

On Thu, Nov 5, 2015 at 11:28 PM, Jeff Holoman <jholoman@cloudera.com> wrote:

> The best thing that I know is the latest javadoc that's committed to trunk:
>
>
> https://github.com/apache/kafka/blob/ef5d168cc8f10ad4f0efe9df4cbe849a4b35496e/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java
>
> Thanks
>
> Jeff
>
>
>
> On Thu, Nov 5, 2015 at 12:51 PM, Cliff Rhyne <crhyne@signal.co> wrote:
>
> > Hi Jeff,
> >
> > Is there a writeup of how to use the new consumer API (either in general
> or
> > for Java)?  I've seen various proposals but I don't see a recent one on
> the
> > actual implementation.  My team wants to start the development work to
> > migrate to 0.9.
> >
> > Thanks,
> > Cliff
> >
> > On Thu, Nov 5, 2015 at 11:18 AM, Jeff Holoman <jholoman@cloudera.com>
> > wrote:
> >
> > > Prabhjot,
> > >
> > > The answer changes slightly for the Producer and Consumer and depends
> on
> > > your timeline and comfort with using new APIs
> > >
> > > Today and in the future, for the Producer, you should be using the
> "new"
> > > producer, which isn't all that new anymore:
> > > org.apache.kafka.clients.producer.KafkaProducer;
> > >
> > >
> > > Today with 0.9 yet to be released you'd likely want to use the
> High-Level
> > > Consumer. This is covered in the official docs here:
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example
> > > and
> > > in this blog post
> > >
> > >
> >
> http://ingest.tips/2014/10/12/kafka-high-level-consumer-frequently-missing-pieces/
> > > along
> > > with most of the other examples that you'll find.
> > >
> > > After .9 is released, I'd encourage you to take a look at the new
> > Consumer
> > > API. This has a lot of advantages in terms of offset management and
> will
> > be
> > > the only consumer client that fully supports security features like SSL
> > > that are slated to be released into the platform.
> > >
> > > Your choice of development language is entirely up to you. Note that
> the
> > > only version of clients that will be maintained in the project going
> > > forward are being implemented in Java, so Scala or Java shouldn't
> matter
> > > too much for you.
> > >
> > > Hope this helps
> > >
> > > Jeff
> > >
> > >
> > > On Thu, Nov 5, 2015 at 12:14 PM, Prabhjot Bharaj <
> prabhbharaj@gmail.com>
> > > wrote:
> > >
> > > > Hello Folks,
> > > >
> > > > Requesting your expertise on this.
> > > > I see that under core/src/main/scala/kafka/producer/, there are many
> > > > implementations - Producer.scala and SyncProducer.scala
> > > >
> > > > Also, going via the producerPerformance.scala, there are 2
> > > implementations
> > > > - NewShinyProducer (which points to KafkaProducer.java) and the
> > > OldProducer
> > > >
> > > > Similar might be the case with Consumers, but I have not seen that
> yet.
> > > >
> > > > Please let me know which producer and consumer is supposed to be used
> > and
> > > > which ones will be phased out in future releases, so I can focus on
> > only
> > > 1
> > > > type of producer and consumer (high level as well as simple)
> > > >
> > > > Thanks,
> > > > Prabhjot
> > > >
> > > > Thanks,
> > > > Prabhjot
> > > >
> > > > On Thu, Nov 5, 2015 at 3:55 PM, Prabhjot Bharaj <
> prabhbharaj@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Adding users as well
> > > > >
> > > > > On Thu, Nov 5, 2015 at 3:37 PM, Prabhjot Bharaj <
> > prabhbharaj@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > >> Hi,
> > > > >>
> > > > >> I'm using the latest update: 0.8.2.2
> > > > >> I would like to use the latest producer and consumer apis
> > > > >> over the past few weeks, I have tried to do some performance
> > > > benchmarking
> > > > >> using the producer and consumer scripts provided in the bin
> > directory.
> > > > It
> > > > >> was a fun activity and I have learnt a lot about kafka.
> > > > >>
> > > > >> But, I have also experienced that sometimes the implementation
of
> > the
> > > > >> performance scripts was not up-to-date or some items were
> different
> > > than
> > > > >> the documentation
> > > > >>
> > > > >> Now, I would like to develop my application with kafka. I'm
> > > comfortable
> > > > >> using scala/java
> > > > >>
> > > > >> Please let me know which producer and consumer (both high level
> and
> > > > >> simple) class/object should I be using
> > > > >>
> > > > >> Thanks a lot,
> > > > >> Prabhjot
> > > > >>
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > ---------------------------------------------------------
> > > > > "There are only 10 types of people in the world: Those who
> understand
> > > > > binary, and those who don't"
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > ---------------------------------------------------------
> > > > "There are only 10 types of people in the world: Those who understand
> > > > binary, and those who don't"
> > > >
> > >
> > >
> > >
> > > --
> > > Jeff Holoman
> > > Systems Engineer
> > >
> >
> >
> >
> > --
> > Cliff Rhyne
> > Software Engineering Lead
> > e: crhyne@signal.co
> > signal.co
> > ________________________
> >
> > Cut Through the Noise
> >
> > This e-mail and any files transmitted with it are for the sole use of the
> > intended recipient(s) and may contain confidential and privileged
> > information. Any unauthorized use of this email is strictly prohibited.
> > ©2015 Signal. All rights reserved.
> >
>
>
>
> --
> Jeff Holoman
> Systems Engineer
>



-- 
---------------------------------------------------------
"There are only 10 types of people in the world: Those who understand
binary, and those who don't"

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message