spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stavros Kontopoulos (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SPARK-24434) Support user-specified driver and executor pod templates
Date Sat, 01 Sep 2018 20:39:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16599728#comment-16599728
] 

Stavros Kontopoulos edited comment on SPARK-24434 at 9/1/18 8:38 PM:
---------------------------------------------------------------------

Spark belongs to the community (no?) and should not serve any company's priorities like Palantirs,
[~mcheah] we don't need the meeting then if we are going to overlap with each other, fine. 
I have a question, why serve Palantir's priorities and not mine? This is not healthy.

You didnt collaborate with me why? .Why nobody respected my will to make it go in 2.4, It
was my priority back then. 

We are implementing the same design, the whole discussion makes no sense, it is not about
mine or your implementation... I am not going to implement the same thing again its not reasonable.
But also I wouldnt create a PR that just works, you can check my comments, that was easy,
just call load and load the template (it is not rocket science this work).

Sorry I dont see any real arguments in the discussion, and as I said I dont want to reply
but the politically correct replies leave me no choice.

We have talked on slack several times and privately as well. You could always have pinged
me but you decided to collaborate on this, without anyone knowing. Probably people don't understand
the meaning of fairness, I am not going to explain it here.  

We can always create any RP we like and then we will see what work is merged, cool.

For good or bad though the meeting has power because k8s committers have the final saying
on merging no? So I dont agree. 

The whole discussion for me is pointless, the message communicated, culture and attitude is
clear. That is for committers or others in the Spark project not to violate the rules fine. 

No rule was violated no worries. 

This is the first time I see this kind of competing "collaboration" among the people of a
group that works for the same cause. It is disappointing.

On the other hand, lesson learned, let's move on, no hard feelings.

 


was (Author: skonto):
Spark belongs to the community (no?) and should not serve any company's priorities like Palantirs,
[~mcheah] we don't need the meeting then if we are going to overlap with each other, fine. 
I have a question, why serve Palantir's priorities and not mine? This is not healthy.

You didnt collaborate with me why? .Why nobody respected my will to make it go in 2.4, It
was my priority back then. 

We are implementing the same design, the whole discussion makes no sense, it is not about
mine or your implementation... I am not going to implement the same thing again its not reasonable.
But also I wouldnt create a PR that just works, you can check my comments, that was easy,
just call load and load the template (it is not rocket science this work).

Sorry I dont see any real arguments in the discussion, and as I said I dont want to reply
but the politically correct replies leave me no choice.

We have talked on slack several times and privately as well. You could always have pinged
me but you decided to collaborate on this, without anyone knowing. Probably people don't understand
the meaning of fairness, I am not going to explain it here.  

We can always create any RP we like and then we will see what work is merged, cool.

For good or bad though the meeting has power because k8s committers have the final saying
on merging no? So I dont agree. 

The whole discussion for me is pointless, the message communicated, culture and attitude is
clear. That is for committers or others in the Spark project not to violate the rules fine. 

No rule was violated no worries. I am also ok.

This is the first time I see this kind of competing "collaboration" among the people of a
group that works for the same cause. It is disappointing.

On the other hand, lesson learned, let's move on, no hard feelings.

 

> Support user-specified driver and executor pod templates
> --------------------------------------------------------
>
>                 Key: SPARK-24434
>                 URL: https://issues.apache.org/jira/browse/SPARK-24434
>             Project: Spark
>          Issue Type: New Feature
>          Components: Kubernetes
>    Affects Versions: 2.4.0
>            Reporter: Yinan Li
>            Priority: Major
>
> With more requests for customizing the driver and executor pods coming, the current approach
of adding new Spark configuration options has some serious drawbacks: 1) it means more Kubernetes
specific configuration options to maintain, and 2) it widens the gap between the declarative
model used by Kubernetes and the configuration model used by Spark. We should start designing
a solution that allows users to specify pod templates as central places for all customization
needs for the driver and executor pods. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message