spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Weichen Xu <weichen...@databricks.com>
Subject Re: Is RDD thread safe?
Date Tue, 12 Nov 2019 05:22:36 GMT
Hi Chang,

RDD/Dataframe is immutable and lazy computed. They are thread safe.

Thanks!

On Tue, Nov 12, 2019 at 12:31 PM Chang Chen <baibaichen@gmail.com> wrote:

> Hi all
>
> I meet a case where I need cache a source RDD, and then create different
> DataFrame from it in different threads to accelerate query.
>
> I know that SparkSession is thread safe(
> https://issues.apache.org/jira/browse/SPARK-15135), but i am not sure
> whether RDD  is thread safe or not
>
> Thanks
>

Mime
View raw message