spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yinan Li (JIRA)" <>
Subject [jira] [Commented] (SPARK-24434) Support user-specified driver and executor pod templates
Date Sat, 02 Jun 2018 23:15:00 GMT


Yinan Li commented on SPARK-24434:

[~skonto] Thanks for the detailed thoughts! I agree with you that we can start with allowing
users to pass in a YAML file that stores the pod template. YAML is more familiar to K8s users
and this is key to make the experience as idiomatic as possible for k8s users, who are the
ones that are aware of what a pod template is and what purpose it serves, and know what they
would like to put into the template.  

> Support user-specified driver and executor pod templates
> --------------------------------------------------------
>                 Key: SPARK-24434
>                 URL:
>             Project: Spark
>          Issue Type: New Feature
>          Components: Kubernetes
>    Affects Versions: 2.4.0
>            Reporter: Yinan Li
>            Priority: Major
> With more requests for customizing the driver and executor pods coming, the current approach
of adding new Spark configuration options has some serious drawbacks: 1) it means more Kubernetes
specific configuration options to maintain, and 2) it widens the gap between the declarative
model used by Kubernetes and the configuration model used by Spark. We should start designing
a solution that allows users to specify pod templates as central places for all customization
needs for the driver and executor pods. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message