spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Saisai Shao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-17637) Packed scheduling for Spark tasks across executors
Date Fri, 23 Sep 2016 08:49:20 GMT

    [ https://issues.apache.org/jira/browse/SPARK-17637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15515836#comment-15515836
] 

Saisai Shao commented on SPARK-17637:
-------------------------------------

[~zhanzhang] would you mind sharing more details about your scheduling mechanism, also does
it specifically bind to dynamic allocation? 

> Packed scheduling for Spark tasks across executors
> --------------------------------------------------
>
>                 Key: SPARK-17637
>                 URL: https://issues.apache.org/jira/browse/SPARK-17637
>             Project: Spark
>          Issue Type: Improvement
>          Components: Scheduler
>            Reporter: Zhan Zhang
>            Priority: Minor
>
> Currently Spark scheduler implements round robin scheduling for tasks to executors. Which
is great as it distributes the load evenly across the cluster, but this leads to significant
resource waste in some cases, especially when dynamic allocation is enabled.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message