spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicholas Chammas <>
Subject Re: Improving volunteer management / JIRAs (split from Spark Improvement Proposals thread)
Date Fri, 07 Oct 2016 21:41:14 GMT
I agree with Cody and others that we need some automation — or at least an
adjusted process — to help us manage organic contributions better.

The objections about automated closing being potentially abrasive are
understood, but I wouldn’t accept that as a defeat for automation. Instead,
it seems like a constraint we should impose on any proposed solution: Make
sure it doesn’t turn contributors off. Rolling as we have been won’t cut
it, and I don’t think adding committers will ever be a sufficient solution
to this particular problem.

To me, it seems like we need a way to filter out viable contributions with
community support from other contributions when it comes to deciding that
automated action is appropriate. Our current tooling isn’t perfect, but
perhaps we can leverage it to create such a filter.

For example, consider the following strawman proposal for how to cut down
on the number of pending but unviable proposals, and simultaneously help
contributors organize to promote viable proposals and get the attention of

   1. Have a bot scan for *stale* JIRA issues and PRs—i.e. they haven’t
   been updated in 20+ days (or D+ days, if you prefer).
   2. Depending on the level of community support, either close the item or
   ping specific people for action. Specifically:
   a. If the JIRA/PR has no input from a committer and the JIRA/PR has 5+
   votes (or V+ votes), ping committers for input. (For PRs, you could
   count comments from different people, or thumbs up on the initial PR post.)
   b. If the JIRA/PR has no input from a committer and the JIRA/PR has less
   than V votes, close it with a gentle message asking the contributor to
   solicit support from either the community or a committer, and try again
   c. If the JIRA/PR has input from a committer or committers, ping them
   for an update.

This is just a rough idea. The point is that when contributors have stale
proposals that they don’t close, committers need to take action. A little
automation to selectively bring contributions to the attention of
committers can perhaps help them manage the backlog of stale contributions.
The “selective” part is implemented in this strawman proposal by using JIRA
votes as a crude proxy for when the community is interested in something,
but it could be anything.

Also, this doesn’t have to be used just to clear out stale proposals. Once
the initial backlog is trimmed down, you could set D to 5 days and use this
as a regular way to bring contributions to the attention of committers.

I dunno if people think this is perhaps too complex, but at our scale I
feel we need some kind of loose but automated system for funneling
contributions through some kind of lifecycle. The status quo is just not
that good (e.g. 474 open PRs <>
against Spark as of this moment).


On Fri, Oct 7, 2016 at 4:48 PM Cody Koeninger <> wrote:

> Matei asked:
> > I agree about empowering people interested here to contribute, but I'm
> wondering, do you think there are technical things that people don't want
> to work on, or is it a matter of what there's been time to do?
> It's a matter of mismanagement and miscommunication.
> The structured streaming kafka jira sat with multiple unanswered
> requests for someone who was a committer to communicate whether they
> were working on it and what the plan was.  I could have done that
> implementation and had it in users' hands months ago.  I didn't
> pre-emptively do it because I didn't want to then have to argue with
> committers about why my code did or did not meet their uncommunicated
> expectations.
> I don't want to re-hash that particular circumstance, I just want to
> make sure it never happens again.
> Hopefully the SIP thread results in clearer expectations, but there
> are still some ideas on the table regarding management of volunteer
> contributions:
> - Closing stale jiras.  I hear the bots are impersonal argument, but
> the alternative of "someone cleans it up" is not sufficient right now
> (with apologies to Sean and all the other janitors).
> - Clear rejection of jiras.  This isn't mean, it's respectful.
> - Clear "I'm working on this", with clear removal and reassignment if
> they go radio silent.  This could be keyed to automated check for
> staleness.
> - Clear expectation that if someone is working on a jira, you can work
> on your own alternative, but you need to communicate.
> I'm sure I've missed some.
> ---------------------------------------------------------------------
> To unsubscribe e-mail:

View raw message