spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Imre Nagi <>
Subject Get Pair of Topic and Message from Kafka + Spark Streaming
Date Wed, 16 Mar 2016 03:58:08 GMT

I'm just trying to process the data that come from the kafka source in my
spark streaming application. What I want to do is get the pair of topic and
message in a tuple from the message stream.

Here is my streams:

 val streams = KafkaUtils.createDirectStream[String, Array[Byte],
> StringDecoder, DefaultDecoder](ssc,kafkaParameter,
>       Array["topic1", "topic2])

I have done several things, but still failed when i did some
transformations from the streams to the pair of topic and message. I hope
somebody can help me here.


View raw message