kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sachin Mittal <sjmit...@gmail.com>
Subject Re: getting intermittent TimeoutException at producer side in streams application
Date Mon, 19 Dec 2016 12:49:45 GMT
Hi,
In order to enable streams logging we would need to add log4j.properties as
part of our classpath right?

Is there any sample log properties we can use.

I think this issue mostly happens when we run streams app in a different DC
than the brokers.

However would like to see in logs if there is any additional information
which we can check.

Thanks
Sachin

On Mon, Dec 19, 2016 at 5:43 PM, Damian Guy <damian.guy@gmail.com> wrote:

> Hi Sachin,
>
> This would usually indicate that may indicate that there is a connectivity
> issue with the brokers. You would need to correlate the logs etc on the
> brokers with the streams logs to try and understand what is happening.
>
> Thanks,
> Damian
>
> On Sun, 18 Dec 2016 at 07:26 Sachin Mittal <sjmittal@gmail.com> wrote:
>
> > Hi all,
> > I have a simple stream application pipeline
> > src.filter.aggragteByKey.mapValues.forEach
> >
> > From time to time I get the following exception:
> > Error sending record to topic test-stream-key-table-changelog
> > org.apache.kafka.common.errors.TimeoutException: Batch containing 2
> > record(s) expired due to timeout while requesting metadata from brokers
> for
> > test-stream-key-table-changelog-0
> >
> > What could be causing the issue?
> > I investigated a bit and saw none of the stage takes a long time. Even in
> > forEach stage where we commit the output to external db takes sub 100 ms
> in
> > worst case.
> >
> > I have right now done a workaround of
> > props.put(ProducerConfig.REQUEST_TIMEOUT_MS_CONFIG, 1800000);
> >
> > Increased the default timeout from 30 seconds to 3 minutes.
> >
> > However to dig deep into the issue where can the problem be?
> >
> > Is it that some stage is taking beyond 30 seconds to execute. Or is it
> some
> > network issue where it is taking a long time to connect to broker itself?
> >
> > Any logging that I can enable at the streams side to get more complete
> > stacktraces?
> >
> > Note that issue occurs in bunches. Then everything works fine for a while
> > then these exceptions come in bunch and then it works fine for sometime
> > then again exceptions and so on.
> >
> > Note that my version is kafka_2.10-0.10.0.1.
> >
> > Thanks
> > Sachin
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message