spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stavros Kontopoulos (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SPARK-24434) Support user-specified driver and executor pod templates
Date Sat, 01 Sep 2018 20:25:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16599728#comment-16599728
] 

Stavros Kontopoulos edited comment on SPARK-24434 at 9/1/18 8:24 PM:
---------------------------------------------------------------------

Spark belongs to the community (no?) and should not serve any company's priorities like Palantirs,
[~mcheah] we don't need the meeting then if we are going to overlap with each other, fine. 
I have a question, why serve Palantir's priorities and not mine? This is not healthy :).

You didnt collaborate with me why? :)  We are implementing the same design, the whole discussion
makes no sense, it is not about mine or your implementation... I am not going to implement
the same thing again its not reasonable. But also I wouldnt create a PR that just works, you
can check my comments, that was easy, just call load and load the template (it is not rocket
science this work).

Sorry I dont see any real arguments in the discussion, and as I said I dont want to reply
but replies leave me no choice.

We have talked on slack several times and privately as well. You could always have pinged
me but you decided to collaborate on this, without anyone knowing. Probably people don't understand
the meaning of fairness, I am not going to explain it here.  

We can always create any RP we like and then we will see what work is merged, cool.

For good or bad though the meeting has power because k8s committers have the final saying
on merging no? So I dont agree. 

The whole discussion for me is pointless, the message communicated, culture and attitude is
clear. The only point is for committers or others in the Spark project not to violate the
rules fine. 

No rule was violated no worries. I am also ok. Lesson learned, let's move on.

 


was (Author: skonto):
Spark belongs to the community (no?) and should not serve any company's priorities like Palantirs,
[~mcheah] we don't need the meeting then if we are going to overlap with each other, fine. 
I have question why serve Palantir's priorities and not mine? This is not healthy :).

You didnt collaborate with me why? :)  We are implementing the same design, the whole discussion
makes no sense, it is not about mine or your implementation... I am not going to implement
the same thing again its not reasonable. But also I wouldnt create a PR that just works, you
can check my comments, that was easy, just call load and load the template (it is not rocket
science this work).

Sorry I dont see any real arguments in the discussion, and as I said I dont want to reply
but replies leave me no choice.

We have talked on slack several times and privately as well. You could always have pinged
me but you decided to collaborate on this, without anyone knowing. Probably people don't understand
the meaning of fairness, I am not going to explain it here.  

We can always create any RP we like and then we will see what work is merged, cool.

For good or bad though the meeting has power because k8s committers have the final saying
on merging no? So I dont agree. 

The whole discussion for me is pointless, the message communicated, culture and attitude is
clear. The only point is for committers or others in the Spark project not to violate the
rules fine. 

No rule was violated no worries. I am also ok. Lesson learned, let's move on.

 

> Support user-specified driver and executor pod templates
> --------------------------------------------------------
>
>                 Key: SPARK-24434
>                 URL: https://issues.apache.org/jira/browse/SPARK-24434
>             Project: Spark
>          Issue Type: New Feature
>          Components: Kubernetes
>    Affects Versions: 2.4.0
>            Reporter: Yinan Li
>            Priority: Major
>
> With more requests for customizing the driver and executor pods coming, the current approach
of adding new Spark configuration options has some serious drawbacks: 1) it means more Kubernetes
specific configuration options to maintain, and 2) it widens the gap between the declarative
model used by Kubernetes and the configuration model used by Spark. We should start designing
a solution that allows users to specify pod templates as central places for all customization
needs for the driver and executor pods. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message