spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hyukjin Kwon <gurwls...@gmail.com>
Subject Re: Resolving all JIRAs affecting EOL releases
Date Sun, 19 May 2019 09:39:49 GMT
I will add one more condition for "updated". So, it will additionally avoid
things updated within one year but left open against EOL releases.

project = SPARK
  AND status in (Open, "In Progress", Reopened)
  AND (
    affectedVersion = EMPTY OR
    NOT (affectedVersion in versionMatch("^3.*")
      OR affectedVersion in versionMatch("^2.4.*")
      OR affectedVersion in versionMatch("^2.3.*")
    )
  )
  AND updated <= -52w

https://issues.apache.org/jira/issues/?filter=12344168&jql=project%20%3D%20SPARK%20%0A%20%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%0A%20%20AND%20(%0A%20%20%20%20affectedVersion%20%3D%20EMPTY%20OR%0A%20%20%20%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.3.*%22)%0A%20%20%20%20)%0A%20%20)%0A%20%20AND%20updated%20%3C%3D%20-52w

This still reduces JIRAs under 1000 which I originally targeted.



2019년 5월 19일 (일) 오후 6:08, Sean Owen <srowen@gmail.com>님이 작성:

> I'd only tweak this to perhaps not close JIRAs that have been updated
> recently -- even just avoiding things updated in the last month. For
> example this would close https://issues.apache.org/jira/browse/SPARK-27758 which
> was opened Friday (though, for other reasons it should probably be closed).
> Still I don't mind it under the logic that it has been reported against
> 2.1.0.
>
> On the other hand, I'd go further and close _anything_ not updated in a
> long time, like a year (or 2 if feeling conservative). That is there's
> probably a lot of old cruft out there that wasn't marked with an Affected
> Version, before that was required.
>
> On Sat, May 18, 2019 at 10:48 PM Hyukjin Kwon <gurwls223@gmail.com> wrote:
>
>> Thanks guys.
>>
>> This thread got more than 3 PMC votes without any objection. I slightly
>> edited JQL from Abdeali's suggestion (thanks, Abdeali).
>>
>>
>> JQL:
>>
>> project = SPARK
>>   AND status in (Open, "In Progress", Reopened)
>>   AND (
>>     affectedVersion = EMPTY OR
>>     NOT (affectedVersion in versionMatch("^3.*")
>>       OR affectedVersion in versionMatch("^2.4.*")
>>       OR affectedVersion in versionMatch("^2.3.*")
>>     )
>>   )
>>
>>
>> https://issues.apache.org/jira/issues/?jql=project%20%3D%20SPARK%20%0A%20%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%0A%20%20AND%20(%0A%20%20%20%20affectedVersion%20%3D%20EMPTY%20OR%0A%20%20%20%20NOT%20(affectedVersion%20in%20versionMatch(%22%5E3.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.4.*%22)%0A%20%20%20%20%20%20OR%20affectedVersion%20in%20versionMatch(%22%5E2.3.*%22)%0A%20%20%20%20)%0A%20%20)
>>
>>
>> It means we will resolve all JIRAs that have EOL releases as affected
>> versions, including no version specified in affected versions - this will
>> reduce open JIRAs under 900.
>>
>> Looks I can use a bulk action feature in JIRA. Tomorrow at the similar
>> time, I will
>> - Label those JIRAs as 'bulk-closed'
>> - Resolve them via `Incomplete` status.
>>
>> Please double check the list and let me know if you guys have any concern.
>>
>>
>>
>>
>>
>> 2019년 5월 18일 (토) 오후 12:22, Dongjoon Hyun <dongjoon.hyun@gmail.com>님이
작성:
>>
>>> +1, too.
>>>
>>> Thank you, Hyukjin!
>>>
>>> Bests,
>>> Dongjoon.
>>>
>>>
>>> On Fri, May 17, 2019 at 9:07 AM Imran Rashid
>>> <irashid@cloudera.com.invalid> wrote:
>>>
>>>> +1, thanks for taking this on
>>>>
>>>> On Wed, May 15, 2019 at 7:26 PM Hyukjin Kwon <gurwls223@gmail.com>
>>>> wrote:
>>>>
>>>>> oh, wait. 'Incomplete' can still make sense in this way then.
>>>>> Yes, I am good with 'Incomplete' too.
>>>>>
>>>>> 2019년 5월 16일 (목) 오전 11:24, Hyukjin Kwon <gurwls223@gmail.com>님이
작성:
>>>>>
>>>>>> I actually recently used 'Incomplete'  a bit when the JIRA is
>>>>>> basically too poorly formed (like just copying and pasting an error)
...
>>>>>>
>>>>>> I was thinking about 'Unresolved' status or `Auto Closed' too. I
>>>>>> double checked they can be reopen as well after resolution.
>>>>>>
>>>>>> [image: Screen Shot 2019-05-16 at 10.35.14 AM.png]
>>>>>> [image: Screen Shot 2019-05-16 at 10.35.39 AM.png]
>>>>>>
>>>>>> 2019년 5월 16일 (목) 오전 11:04, Sean Owen <srowen@gmail.com>님이
작성:
>>>>>>
>>>>>>> Agree, anything without an Affected Version should be old enough
to
>>>>>>> time out.
>>>>>>> I might use "Incomplete" or something as the status, as we haven't
>>>>>>> otherwise used that. Maybe that's simpler than a label. But,
anything like
>>>>>>> that sounds good.
>>>>>>>
>>>>>>> On Wed, May 15, 2019 at 8:40 PM Hyukjin Kwon <gurwls223@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> BTW, affected version became a required field (I don't remember
>>>>>>>> when exactly was .. I believe it's around when we work on
Spark 2.3):
>>>>>>>>
>>>>>>>> [image: Screen Shot 2019-05-16 at 10.29.50 AM.png]
>>>>>>>>
>>>>>>>> So, including all EOL versions and affected versions not
specified
>>>>>>>> will roughly work.
>>>>>>>> Using "Cannot Reproduce" as its status and 'bulk-closed'
label
>>>>>>>> makes the best sense to me.
>>>>>>>>
>>>>>>>> Okie. I want to open this roughly for a week before taking
an
>>>>>>>> actual action for this. If there's no more feedback, I will
do as I said ^
>>>>>>>> next week.
>>>>>>>>
>>>>>>>>
>>>>>>>> 2019년 5월 15일 (수) 오후 11:33, Josh Rosen <rosenville@gmail.com>님이
작성:
>>>>>>>>
>>>>>>>>> +1 in favor of some sort of JIRA cleanup.
>>>>>>>>>
>>>>>>>>> My only request is that we attach some sort of 'bulk-closed'
label
>>>>>>>>> to issues that we close via JIRA filter batch operations
(and resolve the
>>>>>>>>> issues as "Timed Out" / "Cannot Reproduce", not "Fixed").
Using a label
>>>>>>>>> makes it easier to audit what was closed, simplifying
the process of
>>>>>>>>> identifying and re-opening valid issues caught in our
dragnet.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, May 15, 2019 at 7:19 AM Sean Owen <srowen@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I gave up looking through JIRAs a long time ago,
so, big respect
>>>>>>>>>> for
>>>>>>>>>> continuing to try to triage them. I am afraid we're
missing a few
>>>>>>>>>> important bug reports in the torrent, but most JIRAs
are not
>>>>>>>>>> well-formed, just questions, stale, or simply things
that won't be
>>>>>>>>>> added. I do think it's important to reflect that
reality, and so
>>>>>>>>>> I'm
>>>>>>>>>> always in favor of more aggressively closing JIRAs.
I think this
>>>>>>>>>> is
>>>>>>>>>> more standard practice, from projects like TensorFlow/Keras,
>>>>>>>>>> pandas,
>>>>>>>>>> etc to just automatically drop Issues that don't
see activity for
>>>>>>>>>> N
>>>>>>>>>> days. We won't do that, but, are probably on the
other hand far
>>>>>>>>>> too
>>>>>>>>>> lax in closing them.
>>>>>>>>>>
>>>>>>>>>> Remember that JIRAs stay searchable and can be reopened,
so it's
>>>>>>>>>> not
>>>>>>>>>> like we lose much information.
>>>>>>>>>>
>>>>>>>>>> I'd close anything that hasn't had activity in 2
years (?), as a
>>>>>>>>>> start.
>>>>>>>>>> I like the idea of closing things that only affect
an EOL release,
>>>>>>>>>> but, many items aren't marked, so may need to cast
the net wider.
>>>>>>>>>>
>>>>>>>>>> I think only then does it make sense to look at bothering
to
>>>>>>>>>> reproduce
>>>>>>>>>> or evaluate the 1000s that will still remain.
>>>>>>>>>>
>>>>>>>>>> On Wed, May 15, 2019 at 4:25 AM Hyukjin Kwon <gurwls223@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>> >
>>>>>>>>>> > Hi all,
>>>>>>>>>> >
>>>>>>>>>> > I would like to propose to resolve all JIRAs
that affects EOL
>>>>>>>>>> releases - 2.2 and below. and affected version
>>>>>>>>>> > not specified. I was rather against this way
and considered
>>>>>>>>>> this as last resort in roughly 3 years ago
>>>>>>>>>> > when we discussed. Now I think we should go
ahead with this.
>>>>>>>>>> See below.
>>>>>>>>>> >
>>>>>>>>>> > I have been talking care of this for so long
time almost every
>>>>>>>>>> day those 3 years. The number of JIRAs
>>>>>>>>>> > keeps increasing and it does never go down.
Now the number is
>>>>>>>>>> going over 2500 JIRAs.
>>>>>>>>>> > Did you guys know? in JIRA, we can only go through
page by page
>>>>>>>>>> up to 1000 items. So, currently we're even
>>>>>>>>>> > having difficulties to go through every JIRA.
We should
>>>>>>>>>> manually filter out and check each.
>>>>>>>>>> > The number is going over the manageable size.
>>>>>>>>>> >
>>>>>>>>>> > I am not suggesting this without anything actually
trying. This
>>>>>>>>>> is what we have tried within my visibility:
>>>>>>>>>> >
>>>>>>>>>> >   1. In roughly 3 years ago, Sean tried to gather
committers
>>>>>>>>>> and even non-committers people to sort
>>>>>>>>>> >     out this number. At that time, we were only
able to keep
>>>>>>>>>> this number as is. After we lost this momentum,
>>>>>>>>>> >     it kept increasing back.
>>>>>>>>>> >   2. At least I scanned _all_ the previous JIRAs
at least more
>>>>>>>>>> than two times and resolved them. Roughly
>>>>>>>>>> >     once a year. The rest of them are mostly
obsolete but not
>>>>>>>>>> enough information to investigate further.
>>>>>>>>>> >   3. I strictly stick to "Contributing to JIRA
Maintenance"
>>>>>>>>>> https://spark.apache.org/contributing.html and
>>>>>>>>>> >     resolve JIRAs.
>>>>>>>>>> >   4. Promoting other people to comment on JIRA
or actively
>>>>>>>>>> resolve them.
>>>>>>>>>> >
>>>>>>>>>> > One of the facts I realised is the increasing
number of
>>>>>>>>>> committers doesn't virtually help this much (although
>>>>>>>>>> > it might be helpful if somebody active in JIRA
becomes a
>>>>>>>>>> committer.)
>>>>>>>>>> >
>>>>>>>>>> > One of the important thing I should note is
that, it's now
>>>>>>>>>> almost pretty difficult to reproduce and test the
>>>>>>>>>> > issues found in EOL releases. We should git
clone, checkout,
>>>>>>>>>> build and test. And then, see if that issue
>>>>>>>>>> > still exists in upstream, and fix. This is non-trivial
overhead.
>>>>>>>>>> >
>>>>>>>>>> > Therefore, I would like to propose resolving
_all_ the JIRAs
>>>>>>>>>> that targets EOL releases - 2.2 and below.
>>>>>>>>>> > Please let me know if anyone has some concerns
or objections.
>>>>>>>>>> >
>>>>>>>>>> > Thanks.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>>>>>>>>
>>>>>>>>>>

Mime
View raw message