kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Holoman <jholo...@cloudera.com>
Subject Re: New and updated producers and consumers
Date Thu, 05 Nov 2015 17:18:01 GMT
Prabhjot,

The answer changes slightly for the Producer and Consumer and depends on
your timeline and comfort with using new APIs

Today and in the future, for the Producer, you should be using the "new"
producer, which isn't all that new anymore:
org.apache.kafka.clients.producer.KafkaProducer;


Today with 0.9 yet to be released you'd likely want to use the High-Level
Consumer. This is covered in the official docs here:
https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example and
in this blog post
http://ingest.tips/2014/10/12/kafka-high-level-consumer-frequently-missing-pieces/
along
with most of the other examples that you'll find.

After .9 is released, I'd encourage you to take a look at the new Consumer
API. This has a lot of advantages in terms of offset management and will be
the only consumer client that fully supports security features like SSL
that are slated to be released into the platform.

Your choice of development language is entirely up to you. Note that the
only version of clients that will be maintained in the project going
forward are being implemented in Java, so Scala or Java shouldn't matter
too much for you.

Hope this helps

Jeff


On Thu, Nov 5, 2015 at 12:14 PM, Prabhjot Bharaj <prabhbharaj@gmail.com>
wrote:

> Hello Folks,
>
> Requesting your expertise on this.
> I see that under core/src/main/scala/kafka/producer/, there are many
> implementations - Producer.scala and SyncProducer.scala
>
> Also, going via the producerPerformance.scala, there are 2 implementations
> - NewShinyProducer (which points to KafkaProducer.java) and the OldProducer
>
> Similar might be the case with Consumers, but I have not seen that yet.
>
> Please let me know which producer and consumer is supposed to be used and
> which ones will be phased out in future releases, so I can focus on only 1
> type of producer and consumer (high level as well as simple)
>
> Thanks,
> Prabhjot
>
> Thanks,
> Prabhjot
>
> On Thu, Nov 5, 2015 at 3:55 PM, Prabhjot Bharaj <prabhbharaj@gmail.com>
> wrote:
>
> > Adding users as well
> >
> > On Thu, Nov 5, 2015 at 3:37 PM, Prabhjot Bharaj <prabhbharaj@gmail.com>
> > wrote:
> >
> >> Hi,
> >>
> >> I'm using the latest update: 0.8.2.2
> >> I would like to use the latest producer and consumer apis
> >> over the past few weeks, I have tried to do some performance
> benchmarking
> >> using the producer and consumer scripts provided in the bin directory.
> It
> >> was a fun activity and I have learnt a lot about kafka.
> >>
> >> But, I have also experienced that sometimes the implementation of the
> >> performance scripts was not up-to-date or some items were different than
> >> the documentation
> >>
> >> Now, I would like to develop my application with kafka. I'm comfortable
> >> using scala/java
> >>
> >> Please let me know which producer and consumer (both high level and
> >> simple) class/object should I be using
> >>
> >> Thanks a lot,
> >> Prabhjot
> >>
> >
> >
> >
> > --
> > ---------------------------------------------------------
> > "There are only 10 types of people in the world: Those who understand
> > binary, and those who don't"
> >
>
>
>
> --
> ---------------------------------------------------------
> "There are only 10 types of people in the world: Those who understand
> binary, and those who don't"
>



-- 
Jeff Holoman
Systems Engineer

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message