spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cody Koeninger <>
Subject Re: Improving volunteer management / JIRAs (split from Spark Improvement Proposals thread)
Date Fri, 07 Oct 2016 21:59:33 GMT
I really like the idea of using jira votes (and/or watchers?) as a filter!

On Fri, Oct 7, 2016 at 4:41 PM, Nicholas Chammas
<> wrote:
> I agree with Cody and others that we need some automation — or at least an
> adjusted process — to help us manage organic contributions better.
> The objections about automated closing being potentially abrasive are
> understood, but I wouldn’t accept that as a defeat for automation. Instead,
> it seems like a constraint we should impose on any proposed solution: Make
> sure it doesn’t turn contributors off. Rolling as we have been won’t cut it,
> and I don’t think adding committers will ever be a sufficient solution to
> this particular problem.
> To me, it seems like we need a way to filter out viable contributions with
> community support from other contributions when it comes to deciding that
> automated action is appropriate. Our current tooling isn’t perfect, but
> perhaps we can leverage it to create such a filter.
> For example, consider the following strawman proposal for how to cut down on
> the number of pending but unviable proposals, and simultaneously help
> contributors organize to promote viable proposals and get the attention of
> committers:
> Have a bot scan for stale JIRA issues and PRs—i.e. they haven’t been updated
> in 20+ days (or D+ days, if you prefer).
> Depending on the level of community support, either close the item or ping
> specific people for action. Specifically:
> a. If the JIRA/PR has no input from a committer and the JIRA/PR has 5+ votes
> (or V+ votes), ping committers for input. (For PRs, you could count comments
> from different people, or thumbs up on the initial PR post.)
> b. If the JIRA/PR has no input from a committer and the JIRA/PR has less
> than V votes, close it with a gentle message asking the contributor to
> solicit support from either the community or a committer, and try again
> later.
> c. If the JIRA/PR has input from a committer or committers, ping them for an
> update.
> This is just a rough idea. The point is that when contributors have stale
> proposals that they don’t close, committers need to take action. A little
> automation to selectively bring contributions to the attention of committers
> can perhaps help them manage the backlog of stale contributions. The
> “selective” part is implemented in this strawman proposal by using JIRA
> votes as a crude proxy for when the community is interested in something,
> but it could be anything.
> Also, this doesn’t have to be used just to clear out stale proposals. Once
> the initial backlog is trimmed down, you could set D to 5 days and use this
> as a regular way to bring contributions to the attention of committers.
> I dunno if people think this is perhaps too complex, but at our scale I feel
> we need some kind of loose but automated system for funneling contributions
> through some kind of lifecycle. The status quo is just not that good (e.g.
> 474 open PRs against Spark as of this moment).
> Nick
> On Fri, Oct 7, 2016 at 4:48 PM Cody Koeninger <> wrote:
>> Matei asked:
>> > I agree about empowering people interested here to contribute, but I'm
>> > wondering, do you think there are technical things that people don't want to
>> > work on, or is it a matter of what there's been time to do?
>> It's a matter of mismanagement and miscommunication.
>> The structured streaming kafka jira sat with multiple unanswered
>> requests for someone who was a committer to communicate whether they
>> were working on it and what the plan was.  I could have done that
>> implementation and had it in users' hands months ago.  I didn't
>> pre-emptively do it because I didn't want to then have to argue with
>> committers about why my code did or did not meet their uncommunicated
>> expectations.
>> I don't want to re-hash that particular circumstance, I just want to
>> make sure it never happens again.
>> Hopefully the SIP thread results in clearer expectations, but there
>> are still some ideas on the table regarding management of volunteer
>> contributions:
>> - Closing stale jiras.  I hear the bots are impersonal argument, but
>> the alternative of "someone cleans it up" is not sufficient right now
>> (with apologies to Sean and all the other janitors).
>> - Clear rejection of jiras.  This isn't mean, it's respectful.
>> - Clear "I'm working on this", with clear removal and reassignment if
>> they go radio silent.  This could be keyed to automated check for
>> staleness.
>> - Clear expectation that if someone is working on a jira, you can work
>> on your own alternative, but you need to communicate.
>> I'm sure I've missed some.
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail:

To unsubscribe e-mail:

View raw message