On Mon, Mar 18, 2019 at 4:54 PM DB Tsai <email@example.com> wrote:
Rc6 was failed due to the critical bug Xiao mentioned. Rc8 was cut to include those fixes, but I ran into some nexus infra issues when I tried to upload the jars into Apache repo.
The infra team reordered the key servers to prevent the timeout, and rebooted the server. I am running the build again to see if I can successfully upload Spark jars this time. Thanks!
i am having a bit trouble following the RCs this time. is RC6 still the one being voted on? i see on github v2.4.1-rc7 and v2.4.1-rc8 but no email threads.
++ Ryan Blue. What is your opinion regarding this Parquet issue?
Which version of Parquet has this bug? Maybe we can downgrade it.
It worked in 2.3. We broke it with 2.4.0 and were informed of that regression late in the 2.4.0 release process. Since we didn't fix it before the 2.4.0 release, it should have been noted as a known issue. To now claim that there is no regression from 2.4.0 is a circular argument denying the existence of a known regression from 2.3.
From https://issues.apache.org/jira/browse/SPARK-25588, I'm reading that:
- this is a Parquet-Avro version conflict thing
- a downstream app wants different versions of Parquet and Avro than
Spark uses, which triggers it
- it doesn't work in 2.4.0
It's not a regression from 2.4.0, which is the immediate question.
There isn't even a Parquet fix available.
But I'm not even seeing why this is excuse-making?
On Sun, Mar 10, 2019 at 8:44 PM Mark Hamstra <firstname.lastname@example.org> wrote:
> Now wait... we created a regression in 2.4.0. Arguably, we should have blocked that release until we had a fix; but the issue came up late in the release process and it looks to me like there wasn't an adequate fix immediately available, so we did something bad and released 2.4.0 with a known regression. Saying that there is now no regression from 2.4 is tautological and no excuse for not taking in a fix -- and it looks like that fix has been waiting for months.
Sent from my iPhone