spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hyukjin Kwon <gurwls...@gmail.com>
Subject Re: [VOTE] Release Spark 3.0.1 (RC3)
Date Wed, 02 Sep 2020 09:14:29 GMT
For a quick correction:

 >   - For Apache Spark 3.1, we are testing R 4.0 on `master` branch,
 >     but we don't have test coverage on `branch-3.0`.
>      So, I'm wondering if Spark 3.0.1 supports R 4.0 without any issue.

I believe we now test SparkR at branch-3.0 with R 4.0 after
https://github.com/apache/spark/commit/56ec5ddcac8233011c17fc7d120a284707f0f712


2020년 9월 2일 (수) 오후 12:47, Dongjoon Hyun <dongjoon.hyun@gmail.com>님이
작성:

> +1.
>
> Thank you all.
>
> I tested the following additionally with OpenJDK 11.0.8.
>
>     - PySpark UT on Python 3.7.7 with Pandas 0.23.2 / PyArrow 0.15.1.
>     - JDBC integration suite
>     - K8s integration suite (except SparkR test)
>       (Minikube: K8s Client v1.18.8, K8s Server v1.17.11)
>
> For SparkR, it would be great if someone can check the current status
> because SparkR was removed from CRAN.
>
>     - I used `R 3.5.2 (2018-12-20)` like 3.0.0 vote, but
> `dev/make-distribution.sh`
>       failed for me when I used the `--r` option. However, since the
> release artifact has SparkR,
>       I believe it's a problem on my env.
>
>     - Jenkins is using `R 3.6.3 (2020-02-29)` and it seems to succeed,
>       but K8s IT seems to fail consistently due to some unknown reasons.
>       (https://github.com/apache/spark/pull/29533)
>       "Run SparkR on simple dataframe.R example *** FAILED ***"
>
>     - For Apache Spark 3.1, we are testing R 4.0 on `master` branch,
>       but we don't have test coverage on `branch-3.0`.
>       So, I'm wondering if Spark 3.0.1 supports R 4.0 without any issue.
>
> In addition to the one which Xiao mentioned, we also had a late arrival on
> the correctness area,
> but it wasn't a registered blocker before the 3.0.1 RC3 vote. Given that,
> it can be a part of 3.0.2.
>
>     - [SPARK-31511][SQL] Make BytesToBytesMap iterators thread-safe
>       https://github.com/apache/spark/pull/28286#issuecomment-682019408
>
> Bests,
> Dongjoon.
>
> On Tue, Sep 1, 2020 at 7:54 AM Xiao Li <lixiao@databricks.com> wrote:
>
>> Want to change my vote to 0, because we are unable to produce an end-user
>> query to hit this bug.
>>
>> Xiao
>>
>> On Mon, Aug 31, 2020 at 12:41 PM Xiao Li <lixiao@databricks.com> wrote:
>>
>>> -1 due to a regression introduced by a fix in 3.0.1.
>>>
>>> See https://github.com/apache/spark/pull/29602
>>>
>>> Xiao
>>>
>>> On Mon, Aug 31, 2020 at 9:26 AM Tom Graves <tgraves_cs@yahoo.com.invalid>
>>> wrote:
>>>
>>>> +1
>>>>
>>>> Tom
>>>>
>>>> On Friday, August 28, 2020, 09:02:31 AM CDT, 郑瑞峰 <ruifengz@foxmail.com>
>>>> wrote:
>>>>
>>>>
>>>> Please vote on releasing the following candidate as Apache Spark
>>>> version 3.0.1.
>>>>
>>>> The vote is open until Sep 2nd at 9AM PST and passes if a majority +1
>>>> PMC votes are cast, with a minimum of 3 +1 votes.
>>>>
>>>> [ ] +1 Release this package as Apache Spark 3.0.1
>>>> [ ] -1 Do not release this package because ...
>>>>
>>>> To learn more about Apache Spark, please see http://spark.apache.org/
>>>>
>>>> There are currently no issues targeting 3.0.1 (try project = SPARK AND
>>>> "Target Version/s" = "3.0.1" AND status in (Open, Reopened, "In Progress"))
>>>>
>>>> The tag to be voted on is v3.0.1-rc3 (commit
>>>> dc04bf53fe821b7a07f817966c6c173f3b3788c6):
>>>> https://github.com/apache/spark/tree/v3.0.1-rc3
>>>>
>>>> The release files, including signatures, digests, etc. can be found at:
>>>> https://dist.apache.org/repos/dist/dev/spark/v3.0.1-rc3-bin/
>>>>
>>>> Signatures used for Spark RCs can be found in this file:
>>>> https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>>
>>>> The staging repository for this release can be found at:
>>>> https://repository.apache.org/content/repositories/orgapachespark-1357/
>>>>
>>>> The documentation corresponding to this release can be found at:
>>>> https://dist.apache.org/repos/dist/dev/spark/v3.0.1-rc3-docs/
>>>>
>>>> The list of bug fixes going into 3.0.1 can be found at the following
>>>> URL:
>>>> https://s.apache.org/q9g2d
>>>>
>>>> This release is using the release script of the tag v3.0.1-rc3.
>>>>
>>>> FAQ
>>>>
>>>>
>>>> =========================
>>>> How can I help test this release?
>>>> =========================
>>>>
>>>> If you are a Spark user, you can help us test this release by taking
>>>> an existing Spark workload and running on this release candidate, then
>>>> reporting any regressions.
>>>>
>>>> If you're working in PySpark you can set up a virtual env and install
>>>> the current RC and see if anything important breaks, in the Java/Scala
>>>> you can add the staging repository to your projects resolvers and test
>>>> with the RC (make sure to clean up the artifact cache before/after so
>>>> you don't end up building with an out of date RC going forward).
>>>>
>>>> ===========================================
>>>> What should happen to JIRA tickets still targeting 3.0.1?
>>>> ===========================================
>>>>
>>>> The current list of open tickets targeted at 3.0.1 can be found at:
>>>> https://issues.apache.org/jira/projects/SPARK and search for "Target
>>>> Version/s" = 3.0.1
>>>>
>>>> Committers should look at those and triage. Extremely important bug
>>>> fixes, documentation, and API tweaks that impact compatibility should
>>>> be worked on immediately. Everything else please retarget to an
>>>> appropriate release.
>>>>
>>>> ==================
>>>> But my bug isn't fixed?
>>>> ==================
>>>>
>>>> In order to make timely releases, we will typically not hold the
>>>> release unless the bug in question is a regression from the previous
>>>> release. That being said, if there is something which is a regression
>>>> that has not been correctly targeted please ping me or a committer to
>>>> help target the issue.
>>>>
>>>>
>>>>
>>>
>>> --
>>> <https://databricks.com/sparkaisummit/north-america>
>>>
>>
>>
>> --
>> <https://databricks.com/sparkaisummit/north-america>
>>
>

Mime
View raw message