kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Rosenberg <...@squareup.com>
Subject Re: Producer.send questions
Date Sat, 24 Aug 2013 08:44:12 GMT

There are several others I've seen that I would have thought would be
retryable (possibly after an exponential backoff delay).  I'm curious about:

QueueFullException (happens if producerType is 'async')
KafkaException (this seems to wrap lots of base conditions, does one have
to sort through the different wrapped exception types?)
MessageSizeTooLargeException (does a batch of messages get treated as a
single message, when checking for message size too large?)

Also, what about my first question, regarding whether it makes sense to
refresh a producer by closing it and restarting it after a failure?



On Fri, Aug 23, 2013 at 9:07 PM, Jun Rao <junrao@gmail.com> wrote:

> For the most part, only SocketExceptions and NotLeaderForPartitionException
> are recoverable. MessageSizeTooLargeException may be recoverable with a
> smaller batch size.
> Thanks,
> Jun
> On Fri, Aug 23, 2013 at 4:09 PM, Jason Rosenberg <jbr@squareup.com> wrote:
> > I'm using the kafka.javaapi.producer.Producer class from a java client.
> >  I'm wondering if it ever makes sense to refresh a producer by stopping
> it
> > and creating a new one, for example in response to a downstream IO error
> > (e.g. a broker got restarted, or a stale socket, etc.).
> >
> > Or should it always be safe to rely on the producer's implementation to
> > manage it's pool of BlockingChannel connections, etc.
> >
> > I'm also interested in trying to understand which exceptions indicate a
> > failed send() request might be retryable (basically anything that doesn't
> > involve a data-dependent problem, like a malformed message, or a message
> > too large, etc.).
> >
> > Unfortunately, the range of Exceptions that can be thrown by the various
> > javaapi methods is not yet well documented.  It would be nice to have
> some
> > notion of whether an exception is the result of a data error, or a
> > transient downstream connection error, etc.
> >
> > Jason
> >

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message