spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dongjoon Hyun <dongjoon.h...@gmail.com>
Subject Re: Spark 2.4.5 RC2 Preparation Status
Date Wed, 29 Jan 2020 18:04:35 GMT
Great. Sean.

Then, what is your criteria to remove the targeting it from 2.4.5?

It doesn't depend on `Who`, right?

Bests,
Dongjoon.


On Wed, Jan 29, 2020 at 9:56 AM Sean Owen <srowen@gmail.com> wrote:

> OK what if anything is in question for 2.4.5? I don't see anything open
> and targeted for it.
> Are we talking about https://issues.apache.org/jira/browse/SPARK-28344 -
> targeted for 2.4.5 but not backported, and a 'correctness' issue?
> Simply: who argues this must hold up 2.4.5, and if so what's the status?
>
> On Wed, Jan 29, 2020 at 11:27 AM Dongjoon Hyun <dongjoon.hyun@gmail.com>
> wrote:
>
>> Hi, Nicholas and all.
>>
>> RC2 is blocked by the community policy on correctness/dataloss issues.
>>
>> I cut the RC1 when there were no correctness/dataloss issue targeting on
>> 2.4.5. However, it fails because one correctness issue (target = 3.0.0) is
>> resolved and the community changes the target to 2.4.5 at the last day of
>> RC1 vote.
>>
>> As of now, there exists 2.4.5 targeting correctness issue. As a release
>> manager, I cannot cut RC2 until there is no correctness/dataloss issue with
>> target=2.4.5. We need to fix it or we need to move the target version to
>> 2.4.6.
>>
>> That's the current situation. I'm trying to follow the existing community
>> policies, but it seems too idealistic for the release. I'm trying to figure
>> out what is the best option for 2.4.5 in the community. Hopefully, we can
>> start RC2 without known risks at least.
>>
>> For non-correctness issues, it's up to the progress and decision on them.
>> Those issues are not blockers.
>>
>> Bests,
>> Dongjoon.
>>
>> On Wed, Jan 29, 2020 at 05:39 Nicholas Marion <nmarion@us.ibm.com> wrote:
>>
>>> Hello,
>>>
>>> Was wondering if RC2 is expected to release soon? Any chance that
>>> https://issues.apache.org/jira/browse/SPARK-30310 could be added to
>>> branch-2.4 as well for 2.4.5 release? Especially since 2.4.x introduced the
>>> bug?
>>>
>>>
>>> Regards,
>>>
>>> *NICHOLAS T. MARION *
>>> IBM Open Data Analytics for z/OS - *CPO* and *Service Team Lead*
>>> ------------------------------
>>> *Phone: *1-845-433-5010 | *Tie-Line: *293-5010
>>> *E-mail:* *nmarion@us.ibm.com* <nmarion@us.ibm.com>
>>> *Find me on:* [image: LinkedIn:
>>> http://www.linkedin.com/in/nicholasmarion]
>>> <http://www.linkedin.com/in/nicholasmarion>
>>> [image: IBM]
>>>
>>> 2455 South Rd
>>> Poughkeepie, New York 12601-5400
>>> United States
>>> [image: IBM Redbooks Silver Author][image: Data Science Foundations -
>>> Level 1]
>>>
>>>
>>> [image: Inactive hide details for Dongjoon Hyun ---01/20/2020 11:27:19
>>> PM---Hi, All. RC2 was scheduled on Today and all RC1 feedbacks s]Dongjoon
>>> Hyun ---01/20/2020 11:27:19 PM---Hi, All. RC2 was scheduled on Today and
>>> all RC1 feedbacks seems to be addressed.
>>>
>>>
>>>
>>> From: Dongjoon Hyun <dongjoon.hyun@gmail.com>
>>> To: dev <dev@spark.apache.org>
>>> Date: 01/20/2020 11:27 PM
>>> Subject: [EXTERNAL] Spark 2.4.5 RC2 Preparation Status
>>> ------------------------------
>>>
>>>
>>>
>>> Hi, All.
>>>
>>> RC2 was scheduled on Today and all RC1 feedbacks seems to be addressed.
>>> However, I'm waiting for another on-going correctness PR.
>>>
>>>     *https://github.com/apache/spark/pull/27233*
>>> <https://github.com/apache/spark/pull/27233>
>>>     [SPARK-29701][SQL] Correct behaviours of group analytical queries
>>> when empty input given
>>>
>>> Unlike the other correctness issues (I sent previsouly), this one is
>>> active enough to make RC2 fail. As we know, Spark 2.4.5 RC1 vote failed
>>> because the correctness patch landed on `master` branch during the RC1 vote
>>> period and there was official requests for backporting.
>>>
>>>     *https://github.com/apache/spark/pull/27229*
>>> <https://github.com/apache/spark/pull/27229>
>>>     [SPARK-29708][SQL][2.4] Correct aggregated values when grouping sets
>>> are duplicated
>>>
>>> It's risk to start RC2 without considering it because VOTE is also
>>> consuming the community resources.
>>>
>>> BTW, if there is another on-going notable PR for 2.4.5 RC1, please reply
>>> to me.
>>>
>>> Thanks,
>>> Dongjoon.
>>>
>>>
>>>

Mime
View raw message