spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chester @work" <ches...@alpinenow.com>
Subject Re: [Spark SQL] SQLContext getOrCreate incorrect behaviour
Date Mon, 21 Dec 2015 16:23:19 GMT
Jerry
    I thought you should not create more than one SparkContext within one Jvm, ...
Chester

Sent from my iPhone

> On Dec 20, 2015, at 2:59 PM, Jerry Lam <chilinglam@gmail.com> wrote:
> 
> Hi Spark developers,
> 
> I found that SQLContext.getOrCreate(sc: SparkContext) does not behave correctly when
a different spark context is provided.
> 
> ```
> val sc = new SparkContext
> val sqlContext =SQLContext.getOrCreate(sc)
> sc.stop
> ...
> 
> val sc2 = new SparkContext
> val sqlContext2 = SQLContext.getOrCreate(sc2)
> sc2.stop
> ```
> 
> The sqlContext2 will reference sc instead of sc2 and therefore, the program will not
work because sc has been stopped. 
> 
> Best Regards,
> 
> Jerry 

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org


Mime
View raw message