spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reynold Xin <>
Subject Re: [VOTE] Designating maintainers for some Spark components
Date Fri, 07 Nov 2014 07:37:33 GMT

Thanks a lot for commenting on this, but I feel we are splitting hairs
here. Matei did mention -1, followed by "or give feedback". The original
process outlined by Matei was exactly about review, rather than fighting.
Nobody wants to spend their energy fighting.  Everybody is doing it to
improve the project.

In particular, quoting you in your email

"Be careful here. "Responsibility" is pretty much a taboo word. All of
Apache is a group of volunteers. People can disappear at any point, which
is why you need multiple (as my fellow Director warned, on your private
list). And multiple people can disappear."

Take a look at this page:

This Project Management Committee Guide outlines the general
***responsibilities*** of PMC members in managing their projects.

Are you suggesting the wording used by the PMC guideline itself is taboo?

On Thu, Nov 6, 2014 at 11:27 PM, Greg Stein <> wrote:

> [last reply for tonite; let others read; and after the next drink or three,
> I shouldn't be replying...]
> On Thu, Nov 6, 2014 at 11:38 PM, Matei Zaharia <>
> wrote:
> > Alright, Greg, I think I understand how Subversion's model is different,
> > which is that the PMC members are all full committers. However, I still
> > think that the model proposed here is purely organizational (how the PMC
> > and committers organize themselves), and in no way changes peoples'
> > ownership or rights.
> That was not my impression, when your proposal said that maintainers need
> to provide "sign-off".
> Okay. Now my next item of feedback starts here:
> > Certainly the reason I proposed it was organizational, to make sure
> > patches get seen by the right people. I believe that every PMC member
> still
> > has the same responsibility for two reasons:
> >
> > 1) The PMC is actually what selects the maintainers, so basically this
> > mechanism is a way for the PMC to make sure certain people review each
> > patch.
> >
> > 2) Code changes are all still made by consensus, where any individual has
> > veto power over the code. The maintainer model mentioned here is only
> meant
> > to make sure that the "experts" in an area get to see each patch *before*
> > it is merged, and choose whether to exercise their veto power.
> >
> > Let me give a simple example, which is a patch to the Spark core public
> > API. Say I'm a maintainer in this API. Without the maintainer model, the
> > decision on the patch would be made as follows:
> >
> > - Any committer could review the patch and merge it
> > - At any point during this process, I (as the main expert on this) could
> > come in and -1 it, or give feedback
> > - In addition, any other committer beyond me is allowed to -1 this patch
> >
> > With the maintainer model, the process is as follows:
> >
> > - Any committer could review the patch and merge it, but they would need
> > to forward it to me (or another core API maintainer) to make sure we also
> > approve
> > - At any point during this process, I could come in and -1 it, or give
> > feedback
> > - In addition, any other committer beyond me is still allowed to -1 this
> > patch
> >
> > The only change in this model is that committers are responsible to
> > forward patches in these areas to certain other committers. If every
> > committer had perfect oversight of the project, they could have also seen
> > every patch to their component on their own, but this list ensures that
> > they see it even if they somehow overlooked it.
> >
> > It's true that technically this model might "gate" development in the
> > sense of adding some latency, but it doesn't "gate" it any more than
> > consensus as a whole does, where any committer (not even PMC member) can
> -1
> > any code change. In fact I believe this will speed development by
> > motivating the maintainers to be active in reviewing their areas and by
> > reducing the chance that mistakes happen that require a revert.
> >
> > I apologize if this wasn't clear in any way, but I do think it's pretty
> > clear in the original wording of the proposal. The sign-off by a
> maintainer
> > is simply an extra step in the merge process, it does *not* mean that
> other
> > committers can't -1 a patch, or that the maintainers get to review all
> > patches, or that they somehow have more "ownership" of the component
> (since
> > they already had the ability to -1). I also wanted to clarify another
> thing
> > -- it seems there is a misunderstanding that only PMC members can be
> > maintainers, but this was not the point; the PMC *assigns* maintainers
> but
> > they can do it out of the whole committer pool (and if we move to
> > separating the PMC from the committers, I fully expect some non-PMC
> > committers to be made maintainers).
> >
> ... and ends here.
> All of that text is about a process for applying Vetoes. ... That is just
> the wrong focus (IMO).
> Back around 2000, in httpd, we ran into vetoes. It was horrible. The
> community suffered. We actually had a face-to-face at one point, flying in
> people from around the US, gathering a bunch of the httpd committers to
> work through some basic problems. The vetoes were flying fast and furious,
> and it was just the wrong dynamic. Discussion and consensus had been thrown
> aside. Trust was absent. Peer relationships were ruined. (tho thankfully,
> our personal relationships never suffered, and that basis helped us pull it
> back together)
> Contrast that with Subversion. We've had some vetoes, yes. But invariably,
> MOST of them would really be considered "woah. -1 on that. let's talk".
> Only a few were about somebody laying down the veto hammer. Outside those
> few, a -1 was always about opening a discussion to fix a particular commit.
> It looks like you are creating a process to apply vetoes. That seems
> backwards.
> It seems like you want a process to ensure that reviews are performed. IMO,
> all committers/PMC members should begin as *trusted*. Why not? You've
> already voted them in as committers/PMCers. So trust them. Trust.
> And that leads to "trust, but verify". The review process. So how about
> creating a workflow that is focused on "what needs to be reviewed" rather
> than "nobody can make changes, unless John says so". ??
> > I hope this clarifies where we're coming from, and why we believe that
> > this still conforms fully with the spirit of Apache (collaborative, open
> > development that anyone can participate in, and meritocracy for project
> > governance). There were some comments made about the maintainers being
> only
> > some kind of list of people without a requirement to review stuff, but as
> > you can see it's the requirement to review that is the main reason I'm
> > proposing this, to ensure we have an automated process for patches to
> > certain components to be seen. If it helps we may be able to change the
> > wording to something like "it is every committer's responsibility to
> > forward patches for a maintained component to that component's
> maintainer",
> > or something like that, instead of using "sign off".
> As I've mentioned "sign off" is a term for unequal rights. So yes...
> finding a modification would be great. But honestly, I think it would be
> nice to find a workflow that establishes the *reviews* that you're seeking.
> [ Subversion revision props could be used to tag if/when somebody has
> reviewed a particular revision; I dunno if you guys are using svn or git ]
> > If we don't do this, I'd actually be against any measure that lists some
> > component "maintainers" without them having a specific responsibility.
> Be careful here. "Responsibility" is pretty much a taboo word. All of
> Apache is a group of volunteers. People can disappear at any point, which
> is why you need multiple (as my fellow Director warned, on your private
> list). And multiple people can disappear.
> This is the primary reason why Apache prefers lazy consensus. Participants
> may disappear, so everybody should be aware of that, and be aware that lazy
> consensus is in operation. Placing a *volunteer* in the path of forward
> progress is fraught with disaster, in the long run :-)
> >...
> Cheers,
> -g

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message