spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From DB Tsai <dbt...@dbtsai.com.INVALID>
Subject Re: [VOTE] Release Apache Spark 2.4.1 (RC6)
Date Mon, 18 Mar 2019 07:53:48 GMT
Rc6 was failed due to the critical bug Xiao mentioned. Rc8 was cut to
include those fixes, but I ran into some nexus infra issues when I tried to
upload the jars into Apache repo.
https://issues.apache.org/jira/plugins/servlet/mobile#issue/INFRA-18018

The infra team reordered the key servers to prevent the timeout, and
rebooted the server. I am running the build again to see if I can
successfully upload Spark jars this time. Thanks!

On Sun, Mar 17, 2019 at 2:32 PM Koert Kuipers <koert@tresata.com> wrote:

> hi all,
> i am having a bit trouble following the RCs this time. is RC6 still the
> one being voted on? i see on github v2.4.1-rc7 and v2.4.1-rc8 but no email
> threads.
> thanks!
> best, koert
>
> On Mon, Mar 11, 2019 at 1:32 AM Xiao Li <lixiao@databricks.com> wrote:
>
>> ++ Ryan Blue. What is your opinion regarding this Parquet issue?
>>
>> Thanks,
>>
>> Xiao
>>
>> On Sun, Mar 10, 2019 at 9:58 PM Wenchen Fan <cloud0fan@gmail.com> wrote:
>>
>>> Which version of Parquet has this bug? Maybe we can downgrade it.
>>>
>>> On Mon, Mar 11, 2019 at 10:34 AM Mark Hamstra <mark@clearstorydata.com>
>>> wrote:
>>>
>>>> It worked in 2.3. We broke it with 2.4.0 and were informed of that
>>>> regression late in the 2.4.0 release process. Since we didn't fix it before
>>>> the 2.4.0 release, it should have been noted as a known issue. To now claim
>>>> that there is no regression from 2.4.0 is a circular argument denying the
>>>> existence of a known regression from 2.3.
>>>>
>>>> On Sun, Mar 10, 2019 at 6:53 PM Sean Owen <srowen@gmail.com> wrote:
>>>>
>>>>> From https://issues.apache.org/jira/browse/SPARK-25588, I'm reading
>>>>> that:
>>>>>
>>>>> - this is a Parquet-Avro version conflict thing
>>>>> - a downstream app wants different versions of Parquet and Avro than
>>>>> Spark uses, which triggers it
>>>>> - it doesn't work in 2.4.0
>>>>>
>>>>> It's not a regression from 2.4.0, which is the immediate question.
>>>>> There isn't even a Parquet fix available.
>>>>> But I'm not even seeing why this is excuse-making?
>>>>>
>>>>> On Sun, Mar 10, 2019 at 8:44 PM Mark Hamstra <mark@clearstorydata.com>
>>>>> wrote:
>>>>> >
>>>>> > Now wait... we created a regression in 2.4.0. Arguably, we should
>>>>> have blocked that release until we had a fix; but the issue came up late
in
>>>>> the release process and it looks to me like there wasn't an adequate
fix
>>>>> immediately available, so we did something bad and released 2.4.0 with
a
>>>>> known regression. Saying that there is now no regression from 2.4 is
>>>>> tautological and no excuse for not taking in a fix -- and it looks like
>>>>> that fix has been waiting for months.
>>>>>
>>>>
>>
>> --
>> [image:
>> https://databricks.com/sparkaisummit/north-america?utm_source=email&utm_medium=signature]
>>
> --
- DB Sent from my iPhone

Mime
View raw message