spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shyam P <shyamabigd...@gmail.com>
Subject how to refresh the loaded non-streaming dataframe for each steaming batch ?
Date Fri, 06 Sep 2019 04:29:37 GMT
Hi,

I am using spark-sql-2.4.1v to streaming in my PoC.

how to refresh the loaded dataframe from hdfs/cassandra table every time
new batch of stream processed ? What is the practice followed in general to
handle this kind of scenario?

Below is the SOF link for more details .

https://stackoverflow.com/questions/57815645/how-to-refresh-the-contents-of-non-streaming-dataframe

Thank you,
Shyam

Mime
View raw message