kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Holoman <jholo...@cloudera.com>
Subject Re: New and updated producers and consumers
Date Thu, 05 Nov 2015 17:58:34 GMT
The best thing that I know is the latest javadoc that's committed to trunk:

https://github.com/apache/kafka/blob/ef5d168cc8f10ad4f0efe9df4cbe849a4b35496e/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java

Thanks

Jeff



On Thu, Nov 5, 2015 at 12:51 PM, Cliff Rhyne <crhyne@signal.co> wrote:

> Hi Jeff,
>
> Is there a writeup of how to use the new consumer API (either in general or
> for Java)?  I've seen various proposals but I don't see a recent one on the
> actual implementation.  My team wants to start the development work to
> migrate to 0.9.
>
> Thanks,
> Cliff
>
> On Thu, Nov 5, 2015 at 11:18 AM, Jeff Holoman <jholoman@cloudera.com>
> wrote:
>
> > Prabhjot,
> >
> > The answer changes slightly for the Producer and Consumer and depends on
> > your timeline and comfort with using new APIs
> >
> > Today and in the future, for the Producer, you should be using the "new"
> > producer, which isn't all that new anymore:
> > org.apache.kafka.clients.producer.KafkaProducer;
> >
> >
> > Today with 0.9 yet to be released you'd likely want to use the High-Level
> > Consumer. This is covered in the official docs here:
> > https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example
> > and
> > in this blog post
> >
> >
> http://ingest.tips/2014/10/12/kafka-high-level-consumer-frequently-missing-pieces/
> > along
> > with most of the other examples that you'll find.
> >
> > After .9 is released, I'd encourage you to take a look at the new
> Consumer
> > API. This has a lot of advantages in terms of offset management and will
> be
> > the only consumer client that fully supports security features like SSL
> > that are slated to be released into the platform.
> >
> > Your choice of development language is entirely up to you. Note that the
> > only version of clients that will be maintained in the project going
> > forward are being implemented in Java, so Scala or Java shouldn't matter
> > too much for you.
> >
> > Hope this helps
> >
> > Jeff
> >
> >
> > On Thu, Nov 5, 2015 at 12:14 PM, Prabhjot Bharaj <prabhbharaj@gmail.com>
> > wrote:
> >
> > > Hello Folks,
> > >
> > > Requesting your expertise on this.
> > > I see that under core/src/main/scala/kafka/producer/, there are many
> > > implementations - Producer.scala and SyncProducer.scala
> > >
> > > Also, going via the producerPerformance.scala, there are 2
> > implementations
> > > - NewShinyProducer (which points to KafkaProducer.java) and the
> > OldProducer
> > >
> > > Similar might be the case with Consumers, but I have not seen that yet.
> > >
> > > Please let me know which producer and consumer is supposed to be used
> and
> > > which ones will be phased out in future releases, so I can focus on
> only
> > 1
> > > type of producer and consumer (high level as well as simple)
> > >
> > > Thanks,
> > > Prabhjot
> > >
> > > Thanks,
> > > Prabhjot
> > >
> > > On Thu, Nov 5, 2015 at 3:55 PM, Prabhjot Bharaj <prabhbharaj@gmail.com
> >
> > > wrote:
> > >
> > > > Adding users as well
> > > >
> > > > On Thu, Nov 5, 2015 at 3:37 PM, Prabhjot Bharaj <
> prabhbharaj@gmail.com
> > >
> > > > wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >> I'm using the latest update: 0.8.2.2
> > > >> I would like to use the latest producer and consumer apis
> > > >> over the past few weeks, I have tried to do some performance
> > > benchmarking
> > > >> using the producer and consumer scripts provided in the bin
> directory.
> > > It
> > > >> was a fun activity and I have learnt a lot about kafka.
> > > >>
> > > >> But, I have also experienced that sometimes the implementation of
> the
> > > >> performance scripts was not up-to-date or some items were different
> > than
> > > >> the documentation
> > > >>
> > > >> Now, I would like to develop my application with kafka. I'm
> > comfortable
> > > >> using scala/java
> > > >>
> > > >> Please let me know which producer and consumer (both high level and
> > > >> simple) class/object should I be using
> > > >>
> > > >> Thanks a lot,
> > > >> Prabhjot
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > ---------------------------------------------------------
> > > > "There are only 10 types of people in the world: Those who understand
> > > > binary, and those who don't"
> > > >
> > >
> > >
> > >
> > > --
> > > ---------------------------------------------------------
> > > "There are only 10 types of people in the world: Those who understand
> > > binary, and those who don't"
> > >
> >
> >
> >
> > --
> > Jeff Holoman
> > Systems Engineer
> >
>
>
>
> --
> Cliff Rhyne
> Software Engineering Lead
> e: crhyne@signal.co
> signal.co
> ________________________
>
> Cut Through the Noise
>
> This e-mail and any files transmitted with it are for the sole use of the
> intended recipient(s) and may contain confidential and privileged
> information. Any unauthorized use of this email is strictly prohibited.
> ©2015 Signal. All rights reserved.
>



-- 
Jeff Holoman
Systems Engineer

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message