kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Garcia <dav...@spiceworks.com>
Subject Re: KafkaStream: puncutuate() never called even when data is received by process()
Date Wed, 23 Nov 2016 19:30:29 GMT
If you are consuming from more than one topic/partition, punctuate is triggered when the “smallest”
time-value changes.  So, if there is a partition that doesn’t have any more messages on
it, it will always have the smallest time-value and that time value won’t change…hence
punctuate never gets called.


On 11/23/16, 1:01 PM, "Matthias J. Sax" <matthias@confluent.io> wrote:

    Your understanding is correct:
    Punctuate is not triggered base on wall-clock time, but based in
    internally tracked "stream time" that is derived from TimestampExtractor.
    Even if you use WallclockTimestampExtractor, "stream time" is only
    advance if there are input records.
    Not sure why punctuate() is not triggered as you say that you do have
    arriving data.
    Can you share your code?
    On 11/23/16 4:48 AM, shahab wrote:
    > Hello,
    > I am using low level processor and I set the context.schedule(10000),
    > assuming that punctuate() method is invoked every 10 sec .
    >  I have set
    > configProperties.put(StreamsConfig.TIMESTAMP_EXTRACTOR_CLASS_CONFIG,
    > WallclockTimestampExtractor.class.getCanonicalName()) )
    > Although data is keep coming to the topology (as I have logged the incoming
    > tuples to process() ),  punctuate() is never executed.
    > What I am missing?
    > best,
    > Shahab

View raw message