kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steven Schlansker <sschlans...@opentable.com>
Subject Re: Chatty StreamThread commit messages
Date Wed, 01 Mar 2017 19:28:15 GMT
Thank you everyone for the feedback!  I filed
https://issues.apache.org/jira/browse/KAFKA-4829

> On Mar 1, 2017, at 8:22 AM, Michael Noll <michael@confluent.io> wrote:
> 
> Good point, Steven.  +1 here.
> 
> On Wed, Mar 1, 2017 at 8:52 AM, Damian Guy <damian.guy@gmail.com> wrote:
> 
>> +1
>> On Wed, 1 Mar 2017 at 07:15, Guozhang Wang <wangguoz@gmail.com> wrote:
>> 
>>> Hey Steven,
>>> 
>>> That is a good question, and I think your proposal makes sense. Could you
>>> file a JIRA for this change to keep track of it?
>>> 
>>> Guozhang
>>> 
>>> On Tue, Feb 28, 2017 at 1:35 PM, Steven Schlansker <
>>> sschlansker@opentable.com> wrote:
>>> 
>>>> Hi everyone, running with Kafka Streams 0.10.2.0, I see this every
>> commit
>>>> interval:
>>>> 
>>>> 2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
>>> o.a.k.s.p.internals.StreamThread
>>>> - stream-thread [StreamThread-1] Committing task StreamTask 1_31
>>>> 2017-02-28T21:27:16.659Z INFO <> [StreamThread-1]
>>> o.a.k.s.p.internals.StreamThread
>>>> - stream-thread [StreamThread-1] Committing task StreamTask 2_31
>>>> 
>>>> We have ~10 tasks in our topology, 4 topics, and 32 partitions per
>> topic.
>>>> This means every commit interval we log a few hundred lines of the
>> above
>>>> which is an order of magnitude chattier than anything else in the log
>>>> during normal operations.
>>>> 
>>>> Does anyone else find the chattiness of this message annoying?  I would
>>>> personally prefer something more like:
>>>> 
>>>> # existing message is fine at TRACE level for diagnostics
>>>> "TRACE o.a.k.s.p.i.StreamThread - Committing task StreamTask 1_31"
>>>> # normal fast case, wrap them all up into one summary line
>>>> "INFO o.a.k.s.p.i.StreamThreads - 64 stream tasks committed in 25ms"
>>>> # some kind of threshold / messaging in case it doesn't complete
>> quickly
>>>> or logs an exception
>>>> "ERROR o.a.k.s.p.i.StreamThread - StreamTask 1_32 did not commit in
>>> 100ms"
>>>> 
>>>> Thoughts?
>>>> 
>>>> 
>>> 
>>> 
>>> --
>>> -- Guozhang
>>> 
>> 


Mime
View raw message