kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From BYEONG-GI KIM <bg...@bluedigm.com>
Subject Re: Question about applicability of Kafka Streams
Date Fri, 27 May 2016 02:35:57 GMT
Thank you very much for the information!
I'll look into it.

Best regards

KIM

2016-05-27 11:31 GMT+09:00 Tobias Adamson <tobias@stargazer.com.sg>:

> Hi Kim
> Would maybe this example work for you?
>
> https://github.com/apache/kafka/tree/trunk/streams/examples/src/main/java/org/apache/kafka/streams/examples/pageview
> <
> https://github.com/apache/kafka/tree/trunk/streams/examples/src/main/java/org/apache/kafka/streams/examples/pageview
> >
>
> It included JSON -> POJO -> JSON steps and could probably be adapted for
> your case?
>
> Regards
> Toby
> > On 27 May 2016, at 10:20 AM, BYEONG-GI KIM <bgkim@bluedigm.com> wrote:
> >
> > Hello.
> >
> > First I thank you so much for the devs since they've been making a great
> > tool as open-source software.
> >
> > I'm considering to apply a new feature of the Kafka, aka Kafka Streams,
> on
> > my simple handler application, which receives monitoring data from
> Collectd
> > and reproduce transformed messages to Kafka Broker(s). For example, I'd
> > want to change the collected message from Collectd like,
> >
> >
> [{"values":[1901474177],"dstypes":["counter"],"dsnames":["value"],"time":1280959128,"interval":10,"host":"
> > leeloo.octo.it
> >
> ","plugin":"cpu","plugin_instance":"0","type":"cpu","type_instance":"idle"}]
> >
> > to my customized alarm message like,
> >
> > {"alarmMsgType":"threshold", "time":1459436400000, "host":"
> leeloo.octo.it
> > ","category":"CPU","type":"IDLE",
> > "detail":"0","alarmLevel":"critical","message":"cpu
> > error","value":"1901474177"}
> >
> > of course, the re-produced message must be sent to Kafka Broker(s).
> >
> > The problem is that, the message(s) from Collectd is Json-formatted so
> that
> > it seems the Kafka Streams processing would become complicated, i.e., it
> > should be JSONParsed from String to JSON and vise versa after transform.
> >
> > Is it suitable to use the Kafka Stream for this kind of application?
> >
> > Any better idea or comments would also really helpful for me. Thanks in
> > advance!
> >
> > Best regards
> >
> > KIM
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message