spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Richard Marscher <rmarsc...@localytics.com>
Subject Re: Spark Fair Scheduler for Spark Streaming - 1.2 and beyond
Date Fri, 15 May 2015 18:19:57 GMT
The doc is a bit confusing IMO, but at least for my application I had to
use a fair pool configuration to get my stages to be scheduled with FAIR.

On Fri, May 15, 2015 at 2:13 PM, Evo Eftimov <evo.eftimov@isecc.com> wrote:

> No pools for the moment – for each of the apps using the straightforward
> way with the spark conf param for scheduling = FAIR
>
>
>
> Spark is running in a Standalone Mode
>
>
>
> Are you saying that Configuring Pools is mandatory to get the FAIR
> scheduling working – from the docs it seemed optional to me
>
>
>
> *From:* Tathagata Das [mailto:tdas@databricks.com]
> *Sent:* Friday, May 15, 2015 6:45 PM
> *To:* Evo Eftimov
> *Cc:* user
> *Subject:* Re: Spark Fair Scheduler for Spark Streaming - 1.2 and beyond
>
>
>
> How are you configuring the fair scheduler pools?
>
>
>
> On Fri, May 15, 2015 at 8:33 AM, Evo Eftimov <evo.eftimov@isecc.com>
> wrote:
>
> I have run / submitted a few Spark Streaming apps configured with Fair
> scheduling on Spark Streaming 1.2.0, however they still run in a FIFO mode.
> Is FAIR scheduling supported at all for Spark Streaming apps and from what
> release / version - e.g. 1.3.1
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Fair-Scheduler-for-Spark-Streaming-1-2-and-beyond-tp22902.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>
>

Mime
View raw message