spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hyukjin Kwon <gurwls...@gmail.com>
Subject Re: Spark 3.1 branch cut 4th Dec?
Date Fri, 27 Nov 2020 01:42:57 GMT
Just as a gentle reminder, we have one week left before the branch-cut.
Please focus on finalizing the works to land in Spark 3.1.0.
After the branch-cut, I will look through the JIRAs and update the fields
correctly, for example, retargeting to 3.2.0 in the JIRAs targeted 3.1.0.

As Dongjoon pointed out, I either don’t expect a delay anymore. After the
branch-cut, we will have to focus on QA and no new features will be landed
to branch-3.1:

Mid Dec 2020 QA period. Focus on bug fixes, tests, stability and docs.
Generally, no new features merged

in order for us to start the RC in

Early Jan 2021 as planned:Early Jan 2021 Release candidates (RC), voting,
etc. until final release passes


I know this is Thanksgiving day now in the US. Hope you guys enjoy the rest
of the holidays.

Thanks!


2020년 11월 21일 (토) 오전 8:15, Hyukjin Kwon <gurwls223@gmail.com>님이 작성:

> Just for the record, I'll stick to the date we documented at
> https://spark.apache.org/versioning-policy.html
>
> Should be best to stick to what we wrote there given they we delayed once
> already.
>
>
> On Sat, 21 Nov 2020, 02:28 Xiao Li, <gatorsmile@gmail.com> wrote:
>
>> Thank you, Ryan!
>>
>> Xiao
>>
>> Dongjoon Hyun <dongjoon.hyun@gmail.com> 于2020年11月20日周五 上午9:20写道:
>>
>>> It sounds great! :)
>>>
>>> Thanks, Ryan.
>>>
>>> On Fri, Nov 20, 2020 at 9:19 AM Ryan Blue <rblue@netflix.com> wrote:
>>>
>>>> I think we should be able to get the CREATE TABLE changes in. Now that
>>>> the main blocker (EXTERNAL) has been decided, it's just a matter of normal
>>>> review comments.
>>>>
>>>> On Fri, Nov 20, 2020 at 9:05 AM Dongjoon Hyun <dongjoon.hyun@gmail.com>
>>>> wrote:
>>>>
>>>>> Thank you for sharing, Xiao.
>>>>>
>>>>> I hope we are able to make some agreement for CREATE TABLE DDLs, too.
>>>>>
>>>>> Bests,
>>>>> Dongjoon.
>>>>>
>>>>> On Fri, Nov 20, 2020 at 9:01 AM Xiao Li <gatorsmile@gmail.com>
wrote:
>>>>>
>>>>>> https://github.com/apache/spark/pull/28026 is the major feature I
am
>>>>>> tracking. It is painful to keep two sets of CREATE TABLE DDLs with
>>>>>> different behaviors. This hurts the usability of our SQL users, based
on
>>>>>> what I heard. Unfortunately, this PR missed Spark 3.0 release. Now,
I think
>>>>>> we should try our best to address it in 3.1.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Xiao
>>>>>>
>>>>>> Xiao Li <gatorsmile@gmail.com> 于2020年11月20日周五 上午8:52写道:
>>>>>>
>>>>>>> Hi, Dongjoon,
>>>>>>>
>>>>>>> Thank you for your feedback. I think *Early December* does not
mean
>>>>>>> we will cut the branch on Dec 1st. I do not think Dec 1st and
Dec 4th are a
>>>>>>> big deal. Normally, it would be nice to give enough buffer. Based
on my
>>>>>>> understanding, this email is just a *proposal* and a *reminder*.
In
>>>>>>> the past, we often got mixed feedbacks.
>>>>>>>
>>>>>>> Anyway, we are collecting the feedbacks from the whole community.
>>>>>>> Welcome the inputs from everyone else
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Xiao
>>>>>>>
>>>>>>> Dongjoon Hyun <dongjoon.hyun@gmail.com> 于2020年11月20日周五
上午8:33写道:
>>>>>>>
>>>>>>>> Hi, Xiao.
>>>>>>>>
>>>>>>>> I agree.
>>>>>>>>
>>>>>>>>     > Merging the feature work after the branch cut should
not be
>>>>>>>> encouraged in general, although some committers did make
some exceptions
>>>>>>>> based on their own judgement. We should try to avoid merging
the feature
>>>>>>>> work after the branch cut.
>>>>>>>>
>>>>>>>> So, the Apache Spark community accepted your request for
delay
>>>>>>>> already. (Early November to Early December)
>>>>>>>>
>>>>>>>>     -
>>>>>>>> https://github.com/apache/spark-website/commit/0cd0bdc80503882b4737db7e77cc8f9d17ec12ca
>>>>>>>>
>>>>>>>> I don't think the branch cut should be delayed again. We
don't need
>>>>>>>> to have two weeks after Hyukjin's email.
>>>>>>>>
>>>>>>>> Given the delay, I'd strongly recommend to cut the branch
on 1st
>>>>>>>> December.
>>>>>>>>
>>>>>>>> I'll create a `branch-3.1` on 1st December if Hyujkjin is
busy to
>>>>>>>> start to stabilize .
>>>>>>>>
>>>>>>>> Again, it will not block you if you have an exceptional request.
>>>>>>>>
>>>>>>>> However, it would be helpful for all of us if you make it
clear
>>>>>>>> what features you are waiting for now.
>>>>>>>>
>>>>>>>> We are creating Apache Spark together.
>>>>>>>>
>>>>>>>> Bests,
>>>>>>>> Dongjoon.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Nov 19, 2020 at 11:38 PM Xiao Li <gatorsmile@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Correction:
>>>>>>>>>
>>>>>>>>> Merging the feature work after the branch cut should
not be
>>>>>>>>> encouraged in general, although some committers did make
some exceptions
>>>>>>>>> based on their own judgement. We should try to avoid
merging the feature
>>>>>>>>> work after the branch cut.
>>>>>>>>>
>>>>>>>>> This email is a good reminder message. At least, we have
two weeks
>>>>>>>>> ahead of the proposed branch cut date. I hope each feature
owner might
>>>>>>>>> hurry up and try to finish it before the branch cut.
>>>>>>>>>
>>>>>>>>> Xiao
>>>>>>>>>
>>>>>>>>> Xiao Li <gatorsmile@gmail.com> 于2020年11月19日周四
下午11:36写道:
>>>>>>>>>
>>>>>>>>>> We should try to merge the feature work after the
branch cut.
>>>>>>>>>> This should not be encouraged in general, although
some committers did make
>>>>>>>>>> some exceptions based on their own judgement.
>>>>>>>>>>
>>>>>>>>>> This email is a good reminder message. At least,
we have two
>>>>>>>>>> weeks ahead of the proposed branch cut date. I hope
each feature owner
>>>>>>>>>> might hurry up and try to finish it before the branch
cut.
>>>>>>>>>>
>>>>>>>>>> Xiao
>>>>>>>>>>
>>>>>>>>>> Dongjoon Hyun <dongjoon.hyun@gmail.com> 于2020年11月19日周四
下午4:02写道:
>>>>>>>>>>
>>>>>>>>>>> Thank you for your volunteering!
>>>>>>>>>>>
>>>>>>>>>>> Since the previous branch-cuts were always soft-code
freeze
>>>>>>>>>>> which allowed committers to merge to the new
branches still for a while, I
>>>>>>>>>>> believe 1st December will be better for stabilization.
>>>>>>>>>>>
>>>>>>>>>>> Bests,
>>>>>>>>>>> Dongjoon.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Nov 19, 2020 at 3:50 PM Hyukjin Kwon
<
>>>>>>>>>>> gurwls223@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>
>>>>>>>>>>>> I think we haven’t decided yet the exact
branch-cut, code
>>>>>>>>>>>> freeze and release manager.
>>>>>>>>>>>>
>>>>>>>>>>>> As we planned in
>>>>>>>>>>>> https://spark.apache.org/versioning-policy.html
>>>>>>>>>>>>
>>>>>>>>>>>> Early Dec 2020 Code freeze. Release branch
cut
>>>>>>>>>>>>
>>>>>>>>>>>> Code freeze and branch cutting is coming.
>>>>>>>>>>>>
>>>>>>>>>>>> Therefore, we should finish if there are
any remaining works
>>>>>>>>>>>> for Spark 3.1, and
>>>>>>>>>>>> switch to QA mode soon.
>>>>>>>>>>>> I think it’s time to set to keep it on
track, and I would like
>>>>>>>>>>>> to volunteer to help drive this process.
>>>>>>>>>>>>
>>>>>>>>>>>> I am currently thinking 4th Dec as the branch-cut
date.
>>>>>>>>>>>>
>>>>>>>>>>>> Any thoughts?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks all.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>
>>>> --
>>>> Ryan Blue
>>>> Software Engineer
>>>> Netflix
>>>>
>>>

Mime
View raw message