spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sumit Agrawal <su...@continuity1.com>
Subject Spark, read from Kafka stream failing AnalysisException
Date Sun, 05 Apr 2020 15:09:03 GMT
Hello,

I am using Spark 2.4.5, Kafka 2.3.1 on my local machine.

I am able to produce and consume messages on Kafka with bootstrap server config "localhost:9092”

While trying to setup reader with spark streaming API, I am getting an error as 

Exception Message:
Py4JJavaError: An error occurred while calling o166.load.
: org.apache.spark.sql.AnalysisException: Failed to find data source: kafka. Please deploy
the application as per the deployment section of "Structured Streaming + Kafka Integration
Guide".;

Spark Code I am trying to execute: 
df1 = spark.readStream.format("kafka")\
  .option("kafka.bootstrap.servers", "localhost:9092")\
  .option("subscribe", "topic1")\
  .load()

Any guidelines would help.

Regards,
Sumit



---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message