spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kant kodali <kanth...@gmail.com>
Subject Re: is it ok to have multiple sparksession's in one spark structured streaming app?
Date Fri, 08 Sep 2017 18:52:32 GMT
How can I use one SparkSession to talk to both Kafka and Cassandra let's
say?


On Fri, Sep 8, 2017 at 3:46 AM, Arkadiusz Bicz <arkadiusz.bicz@gmail.com>
wrote:

> You don't need multiple spark sessions to have more than one stream
> working, but from maintenance and reliability perspective it is not good
> idea.
>
> On Thu, Sep 7, 2017 at 2:40 AM, kant kodali <kanth909@gmail.com> wrote:
>
>> Hi All,
>>
>> I am wondering if it is ok to have multiple sparksession's in one spark
>> structured streaming app? Basically, I want to create 1) Spark session for
>> reading from Kafka and 2) Another Spark session for storing the mutations
>> of a dataframe/dataset to a persistent table as I get the mutations from
>> #1?
>>
>> Finally, is this a common practice?
>>
>> Thanks,
>> kant
>>
>
>

Mime
View raw message