spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacek Laskowski <>
Subject Re: Akka Stream as the source for Spark Streaming. Please advice...
Date Sat, 12 Nov 2016 14:43:13 GMT

Just to add to Cody's answer...the following snippet works fine on master:

  .option("subscribe", "topic")
  .option("kafka.bootstrap.servers", "localhost:9092")

Don't forget to add spark-sql-kafka-0-10 module to CLASSPATH as follows:

./bin/spark-shell --packages

or libraryDependencies in build.sbt for a standalone Spark app.

See KafkaSourceProvider [1].


Jacek Laskowski
Mastering Apache Spark 2.0
Follow me at

On Thu, Nov 10, 2016 at 8:46 AM, shyla deshpande
<> wrote:
> I am using Spark 2.0.1. I wanted to build a data pipeline using Kafka, Spark
> Streaming and Cassandra using Structured Streaming. But the kafka source
> support for Structured Streaming is not yet available. So now I am trying to
> use Akka Stream as the source to Spark Streaming.
> Want to make sure I am heading in the right direction. Please direct me to
> any sample code and reading material for this.
> Thanks

To unsubscribe e-mail:

View raw message