spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Deepak Sharma <>
Subject Read hdfs files in spark streaming
Date Sun, 09 Jun 2019 13:08:12 GMT
I am using spark streaming application to read from  kafka.
The value coming from kafka message is path to hdfs file.
I am using spark 2.x ,
What is the best way to read this path in spark streaming and then read the
json stored at the hdfs path , may be using , into a df
inside the spark streaming app.
Thanks a lot in advance


View raw message