spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gideon <>
Subject Re: Streaming json records from kafka ... how can I process ... help please :)
Date Wed, 23 Dec 2015 08:05:26 GMT
What you wrote is inaccurate.
When you create a directkafkastream what happens is that you actually create
DirectKafkaInputDStream. This DirectKafkaInputDStream extends a DStream. 2
functions that a DStream has are: map and print
when you map on your DirectKafkaInputDStream what you're actually getting is
a MappedDStream. MappedDStream also extends DStream which means you can
invoke print on it. 
DStream are a Spark Streaming abstraction that allows you to operate on RDDs
in the stream

Regarding the converting the JSON strings, I'm not quite sure what you mean
but you can easily transform the data using the different methods on the
DStream objects that you're getting (like your map example)

I hope that was clear

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message