kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gleb Stsenov <gleb.stse...@gmail.com>
Subject Processor API, how to get last N hours word count
Date Wed, 04 Jul 2018 15:34:06 GMT
Hello,
Just started with Kafka, took 2.0 because it has better unit test support.
Built custom processor, which is basically same as WordCountProcessorDemo
example (github https://goo.gl/XSh7iW ) and can be treated as equal to
that.

Built topology by adding source topic, processor, state store, and sink (to
topic).
Played with KafkaTool and console consumers, can see my wordcounts since
beginning of publisher life.

*Question*:
what would be the correct way to get wordcounts for last 24h?

On topology creation, on processor init, or somehow related to "interactive
queries" feature?
First goal is answering to "what are the most common words during last 24h
(having word count > configured_X)".
Second goal is custom time window. Kinda if word "user" is the most common
in 24h, I want to know its word count for last 36h, or so. Sounds like
"interactive query" for me, but not sure.

Read description of window types in doc, but can't get the idea of applying
them to processor API.

Thank you.
BR,
Gleb.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message