kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tobias Adamson <tob...@stargazer.com.sg>
Subject Re: Question about applicability of Kafka Streams
Date Fri, 27 May 2016 02:31:36 GMT
Hi Kim
Would maybe this example work for you?
https://github.com/apache/kafka/tree/trunk/streams/examples/src/main/java/org/apache/kafka/streams/examples/pageview
<https://github.com/apache/kafka/tree/trunk/streams/examples/src/main/java/org/apache/kafka/streams/examples/pageview>

It included JSON -> POJO -> JSON steps and could probably be adapted for your case?

Regards
Toby
> On 27 May 2016, at 10:20 AM, BYEONG-GI KIM <bgkim@bluedigm.com> wrote:
> 
> Hello.
> 
> First I thank you so much for the devs since they've been making a great
> tool as open-source software.
> 
> I'm considering to apply a new feature of the Kafka, aka Kafka Streams, on
> my simple handler application, which receives monitoring data from Collectd
> and reproduce transformed messages to Kafka Broker(s). For example, I'd
> want to change the collected message from Collectd like,
> 
> [{"values":[1901474177],"dstypes":["counter"],"dsnames":["value"],"time":1280959128,"interval":10,"host":"
> leeloo.octo.it
> ","plugin":"cpu","plugin_instance":"0","type":"cpu","type_instance":"idle"}]
> 
> to my customized alarm message like,
> 
> {"alarmMsgType":"threshold", "time":1459436400000, "host":"leeloo.octo.it
> ","category":"CPU","type":"IDLE",
> "detail":"0","alarmLevel":"critical","message":"cpu
> error","value":"1901474177"}
> 
> of course, the re-produced message must be sent to Kafka Broker(s).
> 
> The problem is that, the message(s) from Collectd is Json-formatted so that
> it seems the Kafka Streams processing would become complicated, i.e., it
> should be JSONParsed from String to JSON and vise versa after transform.
> 
> Is it suitable to use the Kafka Stream for this kind of application?
> 
> Any better idea or comments would also really helpful for me. Thanks in
> advance!
> 
> Best regards
> 
> KIM


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message