spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dongjoon Hyun <dongjoon.h...@gmail.com>
Subject Re: Release Apache Spark 2.4.4 before 3.0.0
Date Tue, 09 Jul 2019 17:11:28 GMT
Thank you for the reply, Sean. Sure. 2.4.x should be a LTS version.

The main reason of 2.4.4 release (before 3.0.0) is to have a better basis
for comparison to 3.0.0.
For example, SPARK-27798 had an old bug, but its correctness issue is only
exposed at Spark 2.4.3.
It would be great if we can have a better basis.

Bests,
Dongjoon.


On Tue, Jul 9, 2019 at 9:52 AM Sean Owen <srowen@gmail.com> wrote:

> We will certainly want a 2.4.4 release eventually. In fact I'd expect
> 2.4.x gets maintained for longer than the usual 18 months, as it's the
> last 2.x branch.
> It doesn't need to happen before 3.0, but could. Usually maintenance
> releases happen 3-4 months apart and the last one was 2 months ago. If
> these are significant issues, sure. It'll probably be August before
> it's out anyway.
>
> On Tue, Jul 9, 2019 at 11:15 AM Dongjoon Hyun <dongjoon.hyun@gmail.com>
> wrote:
> >
> > Hi, All.
> >
> > Spark 2.4.3 was released two months ago (8th May).
> >
> > As of today (9th July), there exist 45 fixes in `branch-2.4` including
> the following correctness or blocker issues.
> >
> >     - SPARK-26038 Decimal toScalaBigInt/toJavaBigInteger not work for
> decimals not fitting in long
> >     - SPARK-26045 Error in the spark 2.4 release package with the
> spark-avro_2.11 dependency
> >     - SPARK-27798 from_avro can modify variables in other rows in local
> mode
> >     - SPARK-27907 HiveUDAF should return NULL in case of 0 rows
> >     - SPARK-28157 Make SHS clear KVStore LogInfo for the blacklist
> entries
> >     - SPARK-28308 CalendarInterval sub-second part should be padded
> before parsing
> >
> > It would be great if we can have Spark 2.4.4 before we are going to get
> busier for 3.0.0.
> > If it's okay, I'd like to volunteer for an 2.4.4 release manager to roll
> it next Monday. (15th July).
> > How do you think about this?
> >
> > Bests,
> > Dongjoon.
>

Mime
View raw message