spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Zhou <zhou10...@gmail.com>
Subject Re: how to refresh the loaded non-streaming dataframe for each steaming batch ?
Date Fri, 06 Sep 2019 18:10:37 GMT
I have the same question with yours

On Thu, Sep 5, 2019 at 9:18 PM Shyam P <shyamabigdata@gmail.com> wrote:

> Hi,
>
> I am using spark-sql-2.4.1v to streaming in my PoC.
>
> how to refresh the loaded dataframe from hdfs/cassandra table every time
> new batch of stream processed ? What is the practice followed in general to
> handle this kind of scenario?
>
> Below is the SOF link for more details .
>
>
> https://stackoverflow.com/questions/57815645/how-to-refresh-the-contents-of-non-streaming-dataframe
>
> Thank you,
> Shyam
>
>

Mime
View raw message