spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stavros Kontopoulos (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SPARK-11714) Make Spark on Mesos honor port restrictions
Date Mon, 04 Jan 2016 00:04:39 GMT

    [ https://issues.apache.org/jira/browse/SPARK-11714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15076309#comment-15076309
] 

Stavros Kontopoulos edited comment on SPARK-11714 at 1/4/16 12:04 AM:
----------------------------------------------------------------------

Before moving on with a PR i was thinking of the following concept:

Check if spark.executor.port is empty if not check if port is within the offered
port range, otherwise refuse offer.
If spark.executor.port it empty this means random port
(default 0). That is a system facility (OS) its not spark convention. For this case pick a
random
port within the offered range.
MesosSchedulerBackend passes to ExecutorInfo the resources offered
thus MesosExecutorBackend could use that info to initialize its port to a specified value.
I am working on this.



was (Author: skonto):
Before moving on with a PR i was thinking of the following concept:

Check if spark.executor.port is empty if not check if port is within the offered
port range, otherwise refuse offer.
If spark.executor.port it empty this means random port
(default 0). That is a system facility (OS) its not spark convention. For this case pick a
random
port within the offered range.
MesosSchedulerBackend passes to ExecutorInfo the resources offered
thus MesosExecutorBackend could use that info to initialize its port to a specified value.



> Make Spark on Mesos honor port restrictions
> -------------------------------------------
>
>                 Key: SPARK-11714
>                 URL: https://issues.apache.org/jira/browse/SPARK-11714
>             Project: Spark
>          Issue Type: Improvement
>          Components: Mesos
>            Reporter: Charles Allen
>
> Currently the MesosSchedulerBackend does not make any effort to honor "ports" as a resource
offer in Mesos. This ask is to have the ports which the executor binds to honor the limits
of the "ports" resource of an offer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message