kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Anderson <manderso...@gmail.com>
Subject Re: ConcurrentModificationException in Kafka Producer 2.1.0
Date Thu, 06 Dec 2018 13:43:33 GMT
Thanks. I assume this will be fixed in 2.1.1?

On Thu, 6 Dec 2018 at 01:11 Guozhang Wang <wangguoz@gmail.com> wrote:

> Hmm.. seems like a regression introduced in since
> "getExpiredInflightBatches".
>
> Actually, just found someone already file a JIRA with a PR:
> https://github.com/apache/kafka/pull/6005/files
>
>
> Guozhang
>
> On Wed, Dec 5, 2018 at 6:54 AM Mark Anderson <manderson23@gmail.com>
> wrote:
>
> > Hi,
> >
> > I'm periodically seeing ConcurrentModificationExceptions in the producer
> > when records are expired e.g.
> >
> > ERROR Dec 05 11:56:13.033 388753 [kafka-producer-network-thread |
> > analogDataProducer] com.x.AnalogMessageBundler Exception
> > org.apache.kafka.common.errors.TimeoutException:
> >  Expiring 1 record(s) for topic-3:1021 ms has passed since batch creation
> > ERROR Dec 05 11:56:13.038 388758 [kafka-producer-network-thread |
> > analogDataProducer] org.apache.kafka.clients.producer.internals.Sender
> > [Producer clientId=analogDataProducer] Uncaught error
> >  in kafka producer I&#x2F;O thread:
> > java.util.ConcurrentModificationException
> >       at java.util.HashMap$HashIterator.nextNode(HashMap.java:1442)
> >         at java.util.HashMap$EntryIterator.next(HashMap.java:1476)
> >             at java.util.HashMap$EntryIterator.next(HashMap.java:1474)
> >                  at
> >
> >
> org.apache.kafka.clients.producer.internals.Sender.getExpiredInflightBatches(Sender.java:177)
> >                       at
> >
> >
> org.apache.kafka.clients.producer.internals.Sender.sendProducerData(Sender.java:353)
> >                               at
> > org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:308)
> >                                    at
> > org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:233)
> >                                         at
> > java.lang.Thread.run(Thread.java:748)
> >
> > Note that I'm continually calling send on the producer (approx 15k
> > calls/second). Previously I've been using 1.1.1 clients and haven't seen
> > this issue.
> >
> > Please let me know if there is any more information I can supply.
> >
> > Thanks,
> > Mark
> >
>
>
> --
> -- Guozhang
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message