kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Kreps <jay.kr...@gmail.com>
Subject Re: 0.8.1 Java Producer API Callbacks
Date Fri, 02 May 2014 17:08:13 GMT
This summary is correct. If you are just starting development now it is
probably reasonable to start with the new producer. We would certainly
appreciate any feedback on it.

My recommendation would be to build the producer off trunk as there were a
few bug fixes since 0.8.1.x that are worth getting.

-Jay


On Fri, May 2, 2014 at 9:49 AM, Neha Narkhede <neha.narkhede@gmail.com>wrote:

> Hi Christian,
>
> As Jun mentioned, the new producer is marked beta since it is new. By 0.9
> or even sooner, we'd expect it to be deployed at LinkedIn and a few other
> companies in a stable state. The APIs, guarantees and features will not
> change by 0.9, just the stability.
>
> Thanks,
> Neha
>
>
> On Fri, May 2, 2014 at 7:50 AM, Jun Rao <junrao@gmail.com> wrote:
>
> > It's beta mostly because it's new.
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Thu, May 1, 2014 at 10:25 PM, cacsar@gmail.com <cacsar@gmail.com>
> > wrote:
> >
> > > Thanks, that's quite helpful. According to this post,
> > > http://blog.empathybox.com/ , it looks like it will be beta then which
> > > seems good enough. Assuming that the beta designation is correct, is
> that
> > > because it won't have as many features/the same flexibility as it's
> > > expected to by 0.9?
> > > Christian
> > >
> > >
> > > On Thu, May 1, 2014 at 9:19 PM, Neha Narkhede <neha.narkhede@gmail.com
> > > >wrote:
> > >
> > > > The javadoc of the new producer is here for now -
> > > >
> > > >
> > >
> >
> http://empathybox.com/kafka-javadoc/index.html?kafka/clients/producer/KafkaProducer.html
> > > >
> > > > Thanks,
> > > > Neha
> > > >
> > > >
> > > > On Thu, May 1, 2014 at 9:16 PM, Jun Rao <junrao@gmail.com> wrote:
> > > >
> > > > > The new producer (that supports callbacks) is in trunk. It will be
> > > > released
> > > > > in 0.8.2. You can look at the java doc of KafkaProducer for the
> api.
> > > > >
> > > > > Thanks
> > > > >
> > > > > Jun
> > > > >
> > > > >
> > > > > On Thu, May 1, 2014 at 8:43 PM, Christian Csar <cacsar@gmail.com>
> > > wrote:
> > > > >
> > > > > > On 05/01/2014 07:22 PM, Christian Csar wrote:
> > > > > > > I'm looking at using the java producer api for 0.8.1 and
I'm
> > > slightly
> > > > > > > confused by this passage from section 4.4 of
> > > > > > > https://kafka.apache.org/documentation.html#theproducer
> > > > > > > "Note that as of Kafka 0.8.1 the async producer does not
have a
> > > > > > > callback, which could be used to register handlers to catch
> send
> > > > > errors.
> > > > > > > Adding such callback functionality is proposed for Kafka
0.9,
> see
> > > > > > > [Proposed Producer
> > > > > > > API](
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Client+Rewrite#ClientRewrite-ProposedProducerAPI
> > > > > )
> > > > > > ."
> > > > > > >
> > > > > > > org.apache.kafka.clients.producer.KafkaProducer in 0.8.1
> appears
> > to
> > > > > have
> > > > > > > public Future<RecordMetadata> send(ProducerRecord
record,
> > Callback
> > > > > > > callback) which looks like the mentioned callback.
> > > > > > >
> > > > > > > How do the callbacks with the async producer? Is it as
> described
> > in
> > > > the
> > > > > > > comment on the send method (see
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/kafka/blob/0.8.1/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java#L151
> > > > > > > for reference)?
> > > > > > >
> > > > > > > Looking around it seems plausible the language in the
> > documentation
> > > > > > > might refer to a separate sort of callback that existed
in 0.7
> > but
> > > > not
> > > > > > > 0.8. In our use case we have something useful to do if
we can
> > > detect
> > > > > > > messages failing to be sent.
> > > > > > >
> > > > > > > Christian
> > > > > > >
> > > > > >
> > > > > > It appears that I was looking at the Java client rather than
the
> > > Scala
> > > > > > java api referenced by the documentation
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/kafka/blob/0.8.1/core/src/main/scala/kafka/javaapi/producer/Producer.scala
> > > > > >
> > > > > > Are both of these currently suited for use from java and still
> > > > > > supported? Given the support for callbacks in the event of
> failure
> > I
> > > am
> > > > > > inclined to use the Java one despite the currently limited
> support
> > > for
> > > > > > specifying partitioners (though it supports specifying the
> > partition)
> > > > or
> > > > > > encoders.
> > > > > >
> > > > > > Any guidance on this would be appreciated.
> > > > > >
> > > > > > Christian
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message