spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arkadiusz Bicz <arkadiusz.b...@gmail.com>
Subject Re: is it ok to have multiple sparksession's in one spark structured streaming app?
Date Fri, 08 Sep 2017 10:46:24 GMT
You don't need multiple spark sessions to have more than one stream
working, but from maintenance and reliability perspective it is not good
idea.

On Thu, Sep 7, 2017 at 2:40 AM, kant kodali <kanth909@gmail.com> wrote:

> Hi All,
>
> I am wondering if it is ok to have multiple sparksession's in one spark
> structured streaming app? Basically, I want to create 1) Spark session for
> reading from Kafka and 2) Another Spark session for storing the mutations
> of a dataframe/dataset to a persistent table as I get the mutations from
> #1?
>
> Finally, is this a common practice?
>
> Thanks,
> kant
>

Mime
View raw message