spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Takeshi Yamamuro <linguin....@gmail.com>
Subject Re: [VOTE] Release Apache Spark 2.4.1 (RC6)
Date Mon, 18 Mar 2019 08:51:02 GMT
Hi,

It might be worth including the fix blow in the next candidate?:
https://github.com/apache/spark/pull/24085


On Mon, Mar 18, 2019 at 4:54 PM DB Tsai <dbtsai@dbtsai.com.invalid> wrote:

> Rc6 was failed due to the critical bug Xiao mentioned. Rc8 was cut to
> include those fixes, but I ran into some nexus infra issues when I tried to
> upload the jars into Apache repo.
> https://issues.apache.org/jira/plugins/servlet/mobile#issue/INFRA-18018
>
> The infra team reordered the key servers to prevent the timeout, and
> rebooted the server. I am running the build again to see if I can
> successfully upload Spark jars this time. Thanks!
>
> On Sun, Mar 17, 2019 at 2:32 PM Koert Kuipers <koert@tresata.com> wrote:
>
>> hi all,
>> i am having a bit trouble following the RCs this time. is RC6 still the
>> one being voted on? i see on github v2.4.1-rc7 and v2.4.1-rc8 but no email
>> threads.
>> thanks!
>> best, koert
>>
>> On Mon, Mar 11, 2019 at 1:32 AM Xiao Li <lixiao@databricks.com> wrote:
>>
>>> ++ Ryan Blue. What is your opinion regarding this Parquet issue?
>>>
>>> Thanks,
>>>
>>> Xiao
>>>
>>> On Sun, Mar 10, 2019 at 9:58 PM Wenchen Fan <cloud0fan@gmail.com> wrote:
>>>
>>>> Which version of Parquet has this bug? Maybe we can downgrade it.
>>>>
>>>> On Mon, Mar 11, 2019 at 10:34 AM Mark Hamstra <mark@clearstorydata.com>
>>>> wrote:
>>>>
>>>>> It worked in 2.3. We broke it with 2.4.0 and were informed of that
>>>>> regression late in the 2.4.0 release process. Since we didn't fix it
before
>>>>> the 2.4.0 release, it should have been noted as a known issue. To now
claim
>>>>> that there is no regression from 2.4.0 is a circular argument denying
the
>>>>> existence of a known regression from 2.3.
>>>>>
>>>>> On Sun, Mar 10, 2019 at 6:53 PM Sean Owen <srowen@gmail.com> wrote:
>>>>>
>>>>>> From https://issues.apache.org/jira/browse/SPARK-25588, I'm reading
>>>>>> that:
>>>>>>
>>>>>> - this is a Parquet-Avro version conflict thing
>>>>>> - a downstream app wants different versions of Parquet and Avro than
>>>>>> Spark uses, which triggers it
>>>>>> - it doesn't work in 2.4.0
>>>>>>
>>>>>> It's not a regression from 2.4.0, which is the immediate question.
>>>>>> There isn't even a Parquet fix available.
>>>>>> But I'm not even seeing why this is excuse-making?
>>>>>>
>>>>>> On Sun, Mar 10, 2019 at 8:44 PM Mark Hamstra <mark@clearstorydata.com>
>>>>>> wrote:
>>>>>> >
>>>>>> > Now wait... we created a regression in 2.4.0. Arguably, we should
>>>>>> have blocked that release until we had a fix; but the issue came
up late in
>>>>>> the release process and it looks to me like there wasn't an adequate
fix
>>>>>> immediately available, so we did something bad and released 2.4.0
with a
>>>>>> known regression. Saying that there is now no regression from 2.4
is
>>>>>> tautological and no excuse for not taking in a fix -- and it looks
like
>>>>>> that fix has been waiting for months.
>>>>>>
>>>>>
>>>
>>> --
>>> [image:
>>> https://databricks.com/sparkaisummit/north-america?utm_source=email&utm_medium=signature]
>>>
>> --
> - DB Sent from my iPhone
>


-- 
---
Takeshi Yamamuro

Mime
View raw message