spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Blue <rb...@netflix.com.INVALID>
Subject Re: Spark 3.1 branch cut 4th Dec?
Date Fri, 20 Nov 2020 17:19:05 GMT
I think we should be able to get the CREATE TABLE changes in. Now that the
main blocker (EXTERNAL) has been decided, it's just a matter of normal
review comments.

On Fri, Nov 20, 2020 at 9:05 AM Dongjoon Hyun <dongjoon.hyun@gmail.com>
wrote:

> Thank you for sharing, Xiao.
>
> I hope we are able to make some agreement for CREATE TABLE DDLs, too.
>
> Bests,
> Dongjoon.
>
> On Fri, Nov 20, 2020 at 9:01 AM Xiao Li <gatorsmile@gmail.com> wrote:
>
>> https://github.com/apache/spark/pull/28026 is the major feature I am
>> tracking. It is painful to keep two sets of CREATE TABLE DDLs with
>> different behaviors. This hurts the usability of our SQL users, based on
>> what I heard. Unfortunately, this PR missed Spark 3.0 release. Now, I think
>> we should try our best to address it in 3.1.
>>
>> Thanks,
>>
>> Xiao
>>
>> Xiao Li <gatorsmile@gmail.com> 于2020年11月20日周五 上午8:52写道:
>>
>>> Hi, Dongjoon,
>>>
>>> Thank you for your feedback. I think *Early December* does not mean we
>>> will cut the branch on Dec 1st. I do not think Dec 1st and Dec 4th are a
>>> big deal. Normally, it would be nice to give enough buffer. Based on my
>>> understanding, this email is just a *proposal* and a *reminder*. In the
>>> past, we often got mixed feedbacks.
>>>
>>> Anyway, we are collecting the feedbacks from the whole community.
>>> Welcome the inputs from everyone else
>>>
>>> Thanks,
>>>
>>> Xiao
>>>
>>> Dongjoon Hyun <dongjoon.hyun@gmail.com> 于2020年11月20日周五 上午8:33写道:
>>>
>>>> Hi, Xiao.
>>>>
>>>> I agree.
>>>>
>>>>     > Merging the feature work after the branch cut should not be
>>>> encouraged in general, although some committers did make some exceptions
>>>> based on their own judgement. We should try to avoid merging the feature
>>>> work after the branch cut.
>>>>
>>>> So, the Apache Spark community accepted your request for delay already.
>>>> (Early November to Early December)
>>>>
>>>>     -
>>>> https://github.com/apache/spark-website/commit/0cd0bdc80503882b4737db7e77cc8f9d17ec12ca
>>>>
>>>> I don't think the branch cut should be delayed again. We don't need to
>>>> have two weeks after Hyukjin's email.
>>>>
>>>> Given the delay, I'd strongly recommend to cut the branch on 1st
>>>> December.
>>>>
>>>> I'll create a `branch-3.1` on 1st December if Hyujkjin is busy to start
>>>> to stabilize .
>>>>
>>>> Again, it will not block you if you have an exceptional request.
>>>>
>>>> However, it would be helpful for all of us if you make it clear what
>>>> features you are waiting for now.
>>>>
>>>> We are creating Apache Spark together.
>>>>
>>>> Bests,
>>>> Dongjoon.
>>>>
>>>>
>>>> On Thu, Nov 19, 2020 at 11:38 PM Xiao Li <gatorsmile@gmail.com> wrote:
>>>>
>>>>> Correction:
>>>>>
>>>>> Merging the feature work after the branch cut should not be encouraged
>>>>> in general, although some committers did make some exceptions based on
>>>>> their own judgement. We should try to avoid merging the feature work
after
>>>>> the branch cut.
>>>>>
>>>>> This email is a good reminder message. At least, we have two weeks
>>>>> ahead of the proposed branch cut date. I hope each feature owner might
>>>>> hurry up and try to finish it before the branch cut.
>>>>>
>>>>> Xiao
>>>>>
>>>>> Xiao Li <gatorsmile@gmail.com> 于2020年11月19日周四 下午11:36写道:
>>>>>
>>>>>> We should try to merge the feature work after the branch cut. This
>>>>>> should not be encouraged in general, although some committers did
make some
>>>>>> exceptions based on their own judgement.
>>>>>>
>>>>>> This email is a good reminder message. At least, we have two weeks
>>>>>> ahead of the proposed branch cut date. I hope each feature owner
might
>>>>>> hurry up and try to finish it before the branch cut.
>>>>>>
>>>>>> Xiao
>>>>>>
>>>>>> Dongjoon Hyun <dongjoon.hyun@gmail.com> 于2020年11月19日周四
下午4:02写道:
>>>>>>
>>>>>>> Thank you for your volunteering!
>>>>>>>
>>>>>>> Since the previous branch-cuts were always soft-code freeze which
>>>>>>> allowed committers to merge to the new branches still for a while,
I
>>>>>>> believe 1st December will be better for stabilization.
>>>>>>>
>>>>>>> Bests,
>>>>>>> Dongjoon.
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Nov 19, 2020 at 3:50 PM Hyukjin Kwon <gurwls223@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> I think we haven’t decided yet the exact branch-cut, code
freeze
>>>>>>>> and release manager.
>>>>>>>>
>>>>>>>> As we planned in https://spark.apache.org/versioning-policy.html
>>>>>>>>
>>>>>>>> Early Dec 2020 Code freeze. Release branch cut
>>>>>>>>
>>>>>>>> Code freeze and branch cutting is coming.
>>>>>>>>
>>>>>>>> Therefore, we should finish if there are any remaining works
for
>>>>>>>> Spark 3.1, and
>>>>>>>> switch to QA mode soon.
>>>>>>>> I think it’s time to set to keep it on track, and I would
like to
>>>>>>>> volunteer to help drive this process.
>>>>>>>>
>>>>>>>> I am currently thinking 4th Dec as the branch-cut date.
>>>>>>>>
>>>>>>>> Any thoughts?
>>>>>>>>
>>>>>>>> Thanks all.
>>>>>>>>
>>>>>>>>

-- 
Ryan Blue
Software Engineer
Netflix

Mime
View raw message