spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Prashant Sharma <scrapco...@gmail.com>
Subject Re: [DISCUSS] Apache Spark 3.0.1 Release
Date Thu, 02 Jul 2020 11:53:18 GMT
Thank you, Holden.

Folks, My health has gone down a bit. So, I will start working on this in a
few days. If this needs to be published sooner, then maybe someone else has
to help out.





On Thu, Jul 2, 2020 at 10:11 AM Holden Karau <holden@pigscanfly.ca> wrote:

> I’m happy to have Prashant do 2.4.7 :)
>
> On Wed, Jul 1, 2020 at 9:40 PM Xiao Li <lixiao@databricks.com> wrote:
>
>> +1 on releasing both 3.0.1 and 2.4.7
>>
>> Great! Three committers volunteer to be a release manager. Ruifeng,
>> Prashant and Holden. Holden just helped release Spark 2.4.6. This time,
>> maybe, Ruifeng and Prashant can be the release manager of 3.0.1 and 2.4.7
>> respectively.
>>
>> Xiao
>>
>> On Wed, Jul 1, 2020 at 2:24 PM Jungtaek Lim <kabhwan.opensource@gmail.com>
>> wrote:
>>
>>> https://issues.apache.org/jira/browse/SPARK-32148 was reported
>>> yesterday, and if the report is valid it looks to be a blocker. I'll try to
>>> take a look sooner.
>>>
>>> On Thu, Jul 2, 2020 at 12:48 AM Shivaram Venkataraman <
>>> shivaram@eecs.berkeley.edu> wrote:
>>>
>>>> Thanks Holden -- it would be great to also get 2.4.7 started
>>>>
>>>> Thanks
>>>> Shivaram
>>>>
>>>> On Tue, Jun 30, 2020 at 10:31 PM Holden Karau <holden@pigscanfly.ca>
>>>> wrote:
>>>> >
>>>> > I can take care of 2.4.7 unless someone else wants to do it.
>>>> >
>>>> > On Tue, Jun 30, 2020 at 8:29 PM Jason Moore <
>>>> Jason.Moore@quantium.com.au> wrote:
>>>> >>
>>>> >> Hi all,
>>>> >>
>>>> >>
>>>> >>
>>>> >> Could I get some input on the severity of this one that I found
>>>> yesterday?  If that’s a correctness issue, should it block this patch?
 Let
>>>> me know under the ticket if there’s more info that I can provide to help.
>>>> >>
>>>> >>
>>>> >>
>>>> >> https://issues.apache.org/jira/browse/SPARK-32136
>>>> >>
>>>> >>
>>>> >>
>>>> >> Thanks,
>>>> >>
>>>> >> Jason.
>>>> >>
>>>> >>
>>>> >>
>>>> >> From: Jungtaek Lim <kabhwan.opensource@gmail.com>
>>>> >> Date: Wednesday, 1 July 2020 at 10:20 am
>>>> >> To: Shivaram Venkataraman <shivaram@eecs.berkeley.edu>
>>>> >> Cc: Prashant Sharma <scrapcodes@gmail.com>, 郑瑞峰 <
>>>> ruifengz@foxmail.com>, Gengliang Wang <gengliang.wang@databricks.com>,
>>>> gurwls223 <gurwls223@gmail.com>, Dongjoon Hyun <dongjoon.hyun@gmail.com>,
>>>> Jules Damji <dmatrix@comcast.net>, Holden Karau <holden@pigscanfly.ca>,
>>>> Reynold Xin <rxin@databricks.com>, Yuanjian Li <xyliyuanjian@gmail.com>,
>>>> "dev@spark.apache.org" <dev@spark.apache.org>, Takeshi Yamamuro <
>>>> linguin.m.s@gmail.com>
>>>> >> Subject: Re: [DISCUSS] Apache Spark 3.0.1 Release
>>>> >>
>>>> >>
>>>> >>
>>>> >> SPARK-32130 [1] looks to be a performance regression introduced
in
>>>> Spark 3.0.0, which is ideal to look into before releasing another bugfix
>>>> version.
>>>> >>
>>>> >>
>>>> >>
>>>> >> 1. https://issues.apache.org/jira/browse/SPARK-32130
>>>> >>
>>>> >>
>>>> >>
>>>> >> On Wed, Jul 1, 2020 at 7:05 AM Shivaram Venkataraman <
>>>> shivaram@eecs.berkeley.edu> wrote:
>>>> >>
>>>> >> Hi all
>>>> >>
>>>> >>
>>>> >>
>>>> >> I just wanted to ping this thread to see if all the outstanding
>>>> blockers for 3.0.1 have been fixed. If so, it would be great if we can get
>>>> the release going. The CRAN team sent us a note that the version SparkR
>>>> available on CRAN for the current R version (4.0.2) is broken and hence we
>>>> need to update the package soon --  it will be great to do it with 3.0.1.
>>>> >>
>>>> >>
>>>> >>
>>>> >> Thanks
>>>> >>
>>>> >> Shivaram
>>>> >>
>>>> >>
>>>> >>
>>>> >> On Wed, Jun 24, 2020 at 8:31 PM Prashant Sharma <
>>>> scrapcodes@gmail.com> wrote:
>>>> >>
>>>> >> +1 for 3.0.1 release.
>>>> >>
>>>> >> I too can help out as release manager.
>>>> >>
>>>> >>
>>>> >>
>>>> >> On Thu, Jun 25, 2020 at 4:58 AM 郑瑞峰 <ruifengz@foxmail.com>
wrote:
>>>> >>
>>>> >> I volunteer to be a release manager of 3.0.1, if nobody is working
>>>> on this.
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >> ------------------ 原始邮件 ------------------
>>>> >>
>>>> >> 发件人: "Gengliang Wang"<gengliang.wang@databricks.com>;
>>>> >>
>>>> >> 发送时间: 2020年6月24日(星期三) 下午4:15
>>>> >>
>>>> >> 收件人: "Hyukjin Kwon"<gurwls223@gmail.com>;
>>>> >>
>>>> >> 抄送: "Dongjoon Hyun"<dongjoon.hyun@gmail.com>;"Jungtaek
Lim"<
>>>> kabhwan.opensource@gmail.com>;"Jules Damji"<dmatrix@comcast.net>;"Holden
>>>> Karau"<holden@pigscanfly.ca>;"Reynold Xin"<rxin@databricks.com>;"Shivaram
>>>> Venkataraman"<shivaram@eecs.berkeley.edu>;"Yuanjian Li"<
>>>> xyliyuanjian@gmail.com>;"Spark dev list"<dev@spark.apache.org>;"Takeshi
>>>> Yamamuro"<linguin.m.s@gmail.com>;
>>>> >>
>>>> >> 主题: Re: [DISCUSS] Apache Spark 3.0.1 Release
>>>> >>
>>>> >>
>>>> >>
>>>> >> +1, the issues mentioned are really serious.
>>>> >>
>>>> >>
>>>> >>
>>>> >> On Tue, Jun 23, 2020 at 7:56 PM Hyukjin Kwon <gurwls223@gmail.com>
>>>> wrote:
>>>> >>
>>>> >> +1.
>>>> >>
>>>> >> Just as a note,
>>>> >> - SPARK-31918 is fixed now, and there's no blocker. - When we build
>>>> SparkR, we should use the latest R version at least 4.0.0+.
>>>> >>
>>>> >>
>>>> >>
>>>> >> 2020년 6월 24일 (수) 오전 11:20, Dongjoon Hyun <dongjoon.hyun@gmail.com>님이
>>>> 작성:
>>>> >>
>>>> >> +1
>>>> >>
>>>> >>
>>>> >>
>>>> >> Bests,
>>>> >>
>>>> >> Dongjoon.
>>>> >>
>>>> >>
>>>> >>
>>>> >> On Tue, Jun 23, 2020 at 1:19 PM Jungtaek Lim <
>>>> kabhwan.opensource@gmail.com> wrote:
>>>> >>
>>>> >> +1 on a 3.0.1 soon.
>>>> >>
>>>> >>
>>>> >>
>>>> >> Probably it would be nice if some Scala experts can take a look
at
>>>> https://issues.apache.org/jira/browse/SPARK-32051 and include the fix
>>>> into 3.0.1 if possible.
>>>> >>
>>>> >> Looks like APIs designed to work with Scala 2.11 & Java bring
>>>> ambiguity in Scala 2.12 & Java.
>>>> >>
>>>> >>
>>>> >>
>>>> >> On Wed, Jun 24, 2020 at 4:52 AM Jules Damji <dmatrix@comcast.net>
>>>> wrote:
>>>> >>
>>>> >> +1 (non-binding)
>>>> >>
>>>> >>
>>>> >>
>>>> >> Sent from my iPhone
>>>> >>
>>>> >> Pardon the dumb thumb typos :)
>>>> >>
>>>> >>
>>>> >>
>>>> >> On Jun 23, 2020, at 11:36 AM, Holden Karau <holden@pigscanfly.ca>
>>>> wrote:
>>>> >>
>>>> >> +1 on a patch release soon
>>>> >>
>>>> >>
>>>> >>
>>>> >> On Tue, Jun 23, 2020 at 10:47 AM Reynold Xin <rxin@databricks.com>
>>>> wrote:
>>>> >>
>>>> >> Error! Filename not specified.
>>>> >>
>>>> >> +1 on doing a new patch release soon. I saw some of these issues
>>>> when preparing the 3.0 release, and some of them are very serious.
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >> On Tue, Jun 23, 2020 at 8:06 AM, Shivaram Venkataraman <
>>>> shivaram@eecs.berkeley.edu> wrote:
>>>> >>
>>>> >> +1 Thanks Yuanjian -- I think it'll be great to have a 3.0.1 release
>>>> soon.
>>>> >>
>>>> >> Shivaram
>>>> >>
>>>> >> On Tue, Jun 23, 2020 at 3:43 AM Takeshi Yamamuro <
>>>> linguin.m.s@gmail.com> wrote:
>>>> >>
>>>> >> Thanks for the heads-up, Yuanjian!
>>>> >>
>>>> >> I also noticed branch-3.0 already has 39 commits after Spark 3.0.0.
>>>> >>
>>>> >> wow, the updates are so quick. Anyway, +1 for the release.
>>>> >>
>>>> >> Bests,
>>>> >> Takeshi
>>>> >>
>>>> >> On Tue, Jun 23, 2020 at 4:59 PM Yuanjian Li <xyliyuanjian@gmail.com>
>>>> wrote:
>>>> >>
>>>> >> Hi dev-list,
>>>> >>
>>>> >> I’m writing this to raise the discussion about Spark 3.0.1
>>>> feasibility since 4 blocker issues were found after Spark 3.0.0:
>>>> >>
>>>> >> [SPARK-31990] The state store compatibility broken will cause a
>>>> correctness issue when Streaming query with `dropDuplicate` uses the
>>>> checkpoint written by the old Spark version.
>>>> >>
>>>> >> [SPARK-32038] The regression bug in handling NaN values in
>>>> COUNT(DISTINCT)
>>>> >>
>>>> >> [SPARK-31918][WIP] CRAN requires to make it working with the latest
>>>> R 4.0. It makes the 3.0 release unavailable on CRAN, and only supports R
>>>> [3.5, 4.0)
>>>> >>
>>>> >> [SPARK-31967] Downgrade vis.js to fix Jobs UI loading time regression
>>>> >>
>>>> >> I also noticed branch-3.0 already has 39 commits after Spark 3.0.0.
>>>> I think it would be great if we have Spark 3.0.1 to deliver the critical
>>>> fixes.
>>>> >>
>>>> >> Any comments are appreciated.
>>>> >>
>>>> >> Best,
>>>> >>
>>>> >> Yuanjian
>>>> >>
>>>> >> --
>>>> >> ---
>>>> >> Takeshi Yamamuro
>>>> >>
>>>> >>
>>>> --------------------------------------------------------------------- To
>>>> unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >> --
>>>> >>
>>>> >> Twitter: https://twitter.com/holdenkarau
>>>> >>
>>>> >> Books (Learning Spark, High Performance Spark, etc.):
>>>> https://amzn.to/2MaRAG9
>>>> >>
>>>> >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>> >
>>>> > --
>>>> > Twitter: https://twitter.com/holdenkarau
>>>> > Books (Learning Spark, High Performance Spark, etc.):
>>>> https://amzn.to/2MaRAG9
>>>> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>>
>>>
>>
>> --
>> <https://databricks.com/sparkaisummit/north-america>
>>
> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9  <https://amzn.to/2MaRAG9>
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>

Mime
View raw message