spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haopu Wang" <HW...@qilinsoft.com>
Subject RE: Is SQLContext thread-safe?
Date Tue, 03 Mar 2015 00:32:16 GMT
Hao, thank you so much for the reply!

Do you already have some JIRA for the discussion?

-----Original Message-----
From: Cheng, Hao [mailto:hao.cheng@intel.com] 
Sent: Tuesday, March 03, 2015 8:23 AM
To: Haopu Wang; user
Subject: RE: Is SQLContext thread-safe?

Currently, each SQLContext has its own configuration, e.g. shuffle
partition number, codegen etc. and it will be shared among the multiple
threads running.

We actually has some internal discussions on this, probably will provide
a "thread local" configuration in the future for a single SQLContext
instance.

-----Original Message-----
From: Haopu Wang [mailto:HWang@qilinsoft.com] 
Sent: Tuesday, March 3, 2015 7:56 AM
To: Cheng, Hao; user
Subject: RE: Is SQLContext thread-safe?

Thanks for the response.

Then I have another question: when will we want to create multiple
SQLContext instances from the same SparkContext? What's the benefit?

-----Original Message-----
From: Cheng, Hao [mailto:hao.cheng@intel.com]
Sent: Monday, March 02, 2015 9:05 PM
To: Haopu Wang; user
Subject: RE: Is SQLContext thread-safe?

Yes it is thread safe, at least it's supposed to be.

-----Original Message-----
From: Haopu Wang [mailto:HWang@qilinsoft.com]
Sent: Monday, March 2, 2015 4:43 PM
To: user
Subject: Is SQLContext thread-safe?

Hi, is it safe to use the same SQLContext to do Select operations in
different threads at the same time? Thank you very much!

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org For additional
commands, e-mail: user-help@spark.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message