spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <so...@cloudera.com>
Subject Re: mesos in spark 2.0.1 - must call stop() otherwise app hangs
Date Thu, 06 Oct 2016 08:18:03 GMT
Yes you should do that. The examples, with one exception, do show this, and
it's always been the intended behavior. I guess it's no surprise to me
because any 'context' object in any framework generally has to be shutdown
for reasons like this.

We need to update the one example. The twist is error handling though,
yeah, because you need to stop() even if an exception occurs. Easy enough
with "finally", but I guess people don't do that. It'd be nice to get rid
of this non-daemon thread if possible

On Thu, Oct 6, 2016 at 9:02 AM Adrian Bridgett <adrian@opensignal.com>
wrote:

> Just one question - what about errors?  Should we be wrapping our entire
> code in a ...finally spark.stop() clause (as per
> http://spark.apache.org/docs/latest/programming-guide.html#unit-testing)?
>
> BTW the .stop() requirement was news to quite a few people here, maybe
> it'd be a good idea to shout more loudly about in the initial
> quickstart/examples?
>
> Cheers,
>
> Adrian
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>

Mime
View raw message