kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Noll <mich...@confluent.io>
Subject Re: Chatty StreamThread commit messages
Date Wed, 01 Mar 2017 16:22:38 GMT
Good point, Steven.  +1 here.

On Wed, Mar 1, 2017 at 8:52 AM, Damian Guy <damian.guy@gmail.com> wrote:

> +1
> On Wed, 1 Mar 2017 at 07:15, Guozhang Wang <wangguoz@gmail.com> wrote:
>
> > Hey Steven,
> >
> > That is a good question, and I think your proposal makes sense. Could you
> > file a JIRA for this change to keep track of it?
> >
> > Guozhang
> >
> > On Tue, Feb 28, 2017 at 1:35 PM, Steven Schlansker <
> > sschlansker@opentable.com> wrote:
> >
> > > Hi everyone, running with Kafka Streams 0.10.2.0, I see this every
> commit
> > > interval:
> > >
> > > 2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
> > o.a.k.s.p.internals.StreamThread
> > > - stream-thread [StreamThread-1] Committing task StreamTask 1_31
> > > 2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
> > o.a.k.s.p.internals.StreamThread
> > > - stream-thread [StreamThread-1] Committing task StreamTask 2_31
> > >
> > > We have ~10 tasks in our topology, 4 topics, and 32 partitions per
> topic.
> > > This means every commit interval we log a few hundred lines of the
> above
> > > which is an order of magnitude chattier than anything else in the log
> > > during normal operations.
> > >
> > > Does anyone else find the chattiness of this message annoying?  I would
> > > personally prefer something more like:
> > >
> > > # existing message is fine at TRACE level for diagnostics
> > > "TRACE o.a.k.s.p.i.StreamThread - Committing task StreamTask 1_31"
> > > # normal fast case, wrap them all up into one summary line
> > > "INFO o.a.k.s.p.i.StreamThreads - 64 stream tasks committed in 25ms"
> > > # some kind of threshold / messaging in case it doesn't complete
> quickly
> > > or logs an exception
> > > "ERROR o.a.k.s.p.i.StreamThread - StreamTask 1_32 did not commit in
> > 100ms"
> > >
> > > Thoughts?
> > >
> > >
> >
> >
> > --
> > -- Guozhang
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message