If you don't send jobs to different pools, then they will all end up in the default pool.  If you leave the intra-pool scheduling policy as the default FIFO, then this will effectively be the same thing as using the default FIFO scheduling.

Depending on what you are trying to accomplish, you need some combination of multiple pools and FAIR scheduling within one or more pools.  And. of course, you need to actually place a job within an appropriate pool.

On Fri, May 15, 2015 at 11:13 AM, Evo Eftimov <evo.eftimov@isecc.com> wrote:

No pools for the moment – for each of the apps using the straightforward way with the spark conf param for scheduling = FAIR

 

Spark is running in a Standalone Mode

 

Are you saying that Configuring Pools is mandatory to get the FAIR scheduling working – from the docs it seemed optional to me

 

From: Tathagata Das [mailto:tdas@databricks.com]
Sent: Friday, May 15, 2015 6:45 PM
To: Evo Eftimov
Cc: user
Subject: Re: Spark Fair Scheduler for Spark Streaming - 1.2 and beyond

 

How are you configuring the fair scheduler pools?

 

On Fri, May 15, 2015 at 8:33 AM, Evo Eftimov <evo.eftimov@isecc.com> wrote:

I have run / submitted a few Spark Streaming apps configured with Fair
scheduling on Spark Streaming 1.2.0, however they still run in a FIFO mode.
Is FAIR scheduling supported at all for Spark Streaming apps and from what
release / version - e.g. 1.3.1




--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Fair-Scheduler-for-Spark-Streaming-1-2-and-beyond-tp22902.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org