kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Garcia <dav...@spiceworks.com>
Subject Re: KafkaStream: puncutuate() never called even when data is received by process()
Date Wed, 23 Nov 2016 19:30:29 GMT
If you are consuming from more than one topic/partition, punctuate is triggered when the “smallest”
time-value changes.  So, if there is a partition that doesn’t have any more messages on
it, it will always have the smallest time-value and that time value won’t change…hence
punctuate never gets called.

-David

On 11/23/16, 1:01 PM, "Matthias J. Sax" <matthias@confluent.io> wrote:

    Your understanding is correct:
    
    Punctuate is not triggered base on wall-clock time, but based in
    internally tracked "stream time" that is derived from TimestampExtractor.
    Even if you use WallclockTimestampExtractor, "stream time" is only
    advance if there are input records.
    
    Not sure why punctuate() is not triggered as you say that you do have
    arriving data.
    
    Can you share your code?
    
    
    
    -Matthias
    
    
    On 11/23/16 4:48 AM, shahab wrote:
    > Hello,
    > 
    > I am using low level processor and I set the context.schedule(10000),
    > assuming that punctuate() method is invoked every 10 sec .
    >  I have set
    > configProperties.put(StreamsConfig.TIMESTAMP_EXTRACTOR_CLASS_CONFIG,
    > WallclockTimestampExtractor.class.getCanonicalName()) )
    > 
    > Although data is keep coming to the topology (as I have logged the incoming
    > tuples to process() ),  punctuate() is never executed.
    > 
    > What I am missing?
    > 
    > best,
    > Shahab
    > 
    
    

Mime
View raw message