spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Evan R. Sparks" <evan.spa...@gmail.com>
Subject Re: SparkContext.stop() ?
Date Fri, 31 Oct 2014 17:39:16 GMT
In cluster settings if you don't explicitly call sc.stop() your application
may hang. Like closing files, network connections, etc, when you're done
with them, it's a good idea to call sc.stop(), which lets the spark master
know that your application is finished consuming resources.

On Fri, Oct 31, 2014 at 10:13 AM, Daniel Siegmann <daniel.siegmann@velos.io>
wrote:

> It is used to shut down the context when you're done with it, but if
> you're using a context for the lifetime of your application I don't think
> it matters.
>
> I use this in my unit tests, because they start up local contexts and you
> can't have multiple local contexts open so each test must stop its context
> when it's done.
>
> On Fri, Oct 31, 2014 at 11:12 AM, ll <duy.huynh.uiv@gmail.com> wrote:
>
>> what is it for?  when do we call it?
>>
>> thanks!
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-user-list.1001560.n3.nabble.com/SparkContext-stop-tp17826.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
>> For additional commands, e-mail: user-help@spark.apache.org
>>
>>
>
>
> --
> Daniel Siegmann, Software Developer
> Velos
> Accelerating Machine Learning
>
> 440 NINTH AVENUE, 11TH FLOOR, NEW YORK, NY 10001
> E: daniel.siegmann@velos.io W: www.velos.io
>

Mime
View raw message