spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hans van den Bogert (JIRA)" <>
Subject [jira] [Commented] (SPARK-4922) Support dynamic allocation for coarse-grained Mesos
Date Tue, 24 Mar 2015 12:07:53 GMT


Hans van den Bogert commented on SPARK-4922:

What were/are the reasons for not having the CoarseGrained backend for mesos deal with executors
like YARN? 

> Support dynamic allocation for coarse-grained Mesos
> ---------------------------------------------------
>                 Key: SPARK-4922
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>          Components: Mesos
>    Affects Versions: 1.2.0
>            Reporter: Andrew Or
>            Priority: Critical
> This brings SPARK-3174, which provided dynamic allocation of cluster resources to Spark
on YARN applications, to Mesos coarse-grained mode. 
> Note that the translation is not as trivial as adding a code path that exposes the request
and kill mechanisms as we did for YARN is SPARK-3822. This is because Mesos coarse-grained
mode schedules on the notion of setting the number of cores allowed for an application (as
in standalone mode) instead of number of executors (as in YARN mode). For more detail, please
see SPARK-4751.
> If you intend to work on this, please provide a detailed design doc!

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message