spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Darabos <daniel.dara...@lynxanalytics.com.INVALID>
Subject Re: [VOTE] Release Apache Spark 2.4.2
Date Thu, 25 Apr 2019 16:50:34 GMT
Right, the Maven packages are cross-published. But
spark-2.4.1-bin-hadoop2.7.tgz was for Scala 2.11
and spark-2.4.2-bin-hadoop2.7.tgz is for Scala 2.12. Since we automatically
pull it in our build, it was a bit of a surprise. :)
Anyway, Scala 2.12 is great. Thanks for the upgrade!

On Thu, Apr 25, 2019 at 3:51 PM Sean Owen <srowen@gmail.com> wrote:

> The default is 2.12 now, but it's cross-published for 2.11 and 2.12 still:
> https://search.maven.org/search?q=g:org.apache.spark
> You can still build for 2.11 from the source as desired. This was sorta
> covered in the 2.4.1 release notes, where 2.11 support was deprecated:
> http://spark.apache.org/releases/spark-release-2-4-1.html
> It's going to be removed in 3.0, note.
>
> On Thu, Apr 25, 2019 at 7:22 AM Daniel Darabos
> <daniel.darabos@lynxanalytics.com.invalid> wrote:
>
>> Looks like this is the first release where the pre-built binaries are for
>> Scala 2.12. For people using the pre-built binaries this is a big deal. We
>> cannot upgrade to Spark 2.4.2 without switching to Scala 2.12, which
>> involves upgrading or replacing a whole bunch of dependencies. (Or we could
>> start building our own Spark, but that's a lot of work for the sake of
>> delaying the inevitable.)
>>
>> I think it would have been better to make this switch on a minor rather
>> than patch version. But this is also fine, it just may be worth
>> highlighting in the release notes.
>>
>> Thanks,
>> Daniel
>>
>> On Wed, Apr 24, 2019 at 4:34 AM Wenchen Fan <cloud0fan@gmail.com> wrote:
>>
>>> This vote passes! I'll follow up with a formal release announcement soon.
>>>
>>> +1:
>>> Michael Armbrust (binding)
>>> Holden Karau (binding)
>>> Marco Gaido
>>> Sean Owen (binding)
>>> Felix Cheung (binding)
>>> vaquar khan
>>> Shixiong(Ryan) Zhu (binding)
>>> Xiao Li (binding)
>>> Wenchen Fan (binding)
>>>
>>> +0
>>> shane knapp
>>>
>>>
>>> On Wed, Apr 24, 2019 at 10:27 AM Wenchen Fan <cloud0fan@gmail.com>
>>> wrote:
>>>
>>>> adding my own +1
>>>>
>>>> On Tue, Apr 23, 2019 at 12:27 PM Xiao Li <lixiao@databricks.com> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> Xiao
>>>>>
>>>>> On Mon, Apr 22, 2019 at 4:56 PM Shixiong(Ryan) Zhu <
>>>>> shixiong@databricks.com> wrote:
>>>>>
>>>>>> +1 I have tested it and looks good!
>>>>>>
>>>>>> Best Regards,
>>>>>> Ryan
>>>>>>
>>>>>>
>>>>>> On Sun, Apr 21, 2019 at 8:49 PM Wenchen Fan <cloud0fan@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Yea these should be mentioned in the 2.4.1 release notes.
>>>>>>>
>>>>>>> It seems we only have one ticket that is labeled as "release-notes"
>>>>>>> for 2.4.2: https://issues.apache.org/jira/browse/SPARK-27419
. I'll
>>>>>>> mention it when I write release notes.
>>>>>>>
>>>>>>> On Mon, Apr 22, 2019 at 5:46 AM Sean Owen <srowen@gmail.com>
wrote:
>>>>>>>
>>>>>>>> One minor comment: for 2.4.1 we had a couple JIRAs marked
>>>>>>>> 'release-notes':
>>>>>>>>
>>>>>>>> https://issues.apache.org/jira/browse/SPARK-27198?jql=project%20%3D%20SPARK%20and%20fixVersion%20%20in%20(2.4.1%2C%202.4.2)%20and%20labels%20%3D%20%27release-notes%27
>>>>>>>>
>>>>>>>> They should be mentioned in
>>>>>>>> https://spark.apache.org/releases/spark-release-2-4-1.html
possibly
>>>>>>>> like "Changes of behavior" in
>>>>>>>> https://spark.apache.org/releases/spark-release-2-4-0.html
>>>>>>>>
>>>>>>>> I can retroactively update that page; is this part of the
notes for
>>>>>>>> the release process though? I missed this one for sure as
it's easy
>>>>>>>> to
>>>>>>>> overlook with all the pages being updated per release.
>>>>>>>>
>>>>>>>> On Thu, Apr 18, 2019 at 9:51 PM Wenchen Fan <cloud0fan@gmail.com>
>>>>>>>> wrote:
>>>>>>>> >
>>>>>>>> > Please vote on releasing the following candidate as
Apache Spark
>>>>>>>> version 2.4.2.
>>>>>>>> >
>>>>>>>> > The vote is open until April 23 PST and passes if a
majority +1
>>>>>>>> PMC votes are cast, with
>>>>>>>> > a minimum of 3 +1 votes.
>>>>>>>> >
>>>>>>>> > [ ] +1 Release this package as Apache Spark 2.4.2
>>>>>>>> > [ ] -1 Do not release this package because ...
>>>>>>>> >
>>>>>>>> > To learn more about Apache Spark, please see
>>>>>>>> http://spark.apache.org/
>>>>>>>> >
>>>>>>>> > The tag to be voted on is v2.4.2-rc1 (commit
>>>>>>>> a44880ba74caab7a987128cb09c4bee41617770a):
>>>>>>>> > https://github.com/apache/spark/tree/v2.4.2-rc1
>>>>>>>> >
>>>>>>>> > The release files, including signatures, digests, etc.
can be
>>>>>>>> found at:
>>>>>>>> > https://dist.apache.org/repos/dist/dev/spark/v2.4.2-rc1-bin/
>>>>>>>> >
>>>>>>>> > Signatures used for Spark RCs can be found in this file:
>>>>>>>> > https://dist.apache.org/repos/dist/dev/spark/KEYS
>>>>>>>> >
>>>>>>>> > The staging repository for this release can be found
at:
>>>>>>>> >
>>>>>>>> https://repository.apache.org/content/repositories/orgapachespark-1322/
>>>>>>>> >
>>>>>>>> > The documentation corresponding to this release can
be found at:
>>>>>>>> > https://dist.apache.org/repos/dist/dev/spark/v2.4.2-rc1-docs/
>>>>>>>> >
>>>>>>>> > The list of bug fixes going into 2.4.1 can be found
at the
>>>>>>>> following URL:
>>>>>>>> > https://issues.apache.org/jira/projects/SPARK/versions/12344996
>>>>>>>> >
>>>>>>>> > FAQ
>>>>>>>> >
>>>>>>>> > =========================
>>>>>>>> > How can I help test this release?
>>>>>>>> > =========================
>>>>>>>> >
>>>>>>>> > If you are a Spark user, you can help us test this release
by
>>>>>>>> taking
>>>>>>>> > an existing Spark workload and running on this release
candidate,
>>>>>>>> then
>>>>>>>> > reporting any regressions.
>>>>>>>> >
>>>>>>>> > If you're working in PySpark you can set up a virtual
env and
>>>>>>>> install
>>>>>>>> > the current RC and see if anything important breaks,
in the
>>>>>>>> Java/Scala
>>>>>>>> > you can add the staging repository to your projects
resolvers and
>>>>>>>> test
>>>>>>>> > with the RC (make sure to clean up the artifact cache
>>>>>>>> before/after so
>>>>>>>> > you don't end up building with a out of date RC going
forward).
>>>>>>>> >
>>>>>>>> > ===========================================
>>>>>>>> > What should happen to JIRA tickets still targeting 2.4.2?
>>>>>>>> > ===========================================
>>>>>>>> >
>>>>>>>> > The current list of open tickets targeted at 2.4.2 can
be found
>>>>>>>> at:
>>>>>>>> > https://issues.apache.org/jira/projects/SPARK and search
for
>>>>>>>> "Target Version/s" = 2.4.2
>>>>>>>> >
>>>>>>>> > Committers should look at those and triage. Extremely
important
>>>>>>>> bug
>>>>>>>> > fixes, documentation, and API tweaks that impact compatibility
>>>>>>>> should
>>>>>>>> > be worked on immediately. Everything else please retarget
to an
>>>>>>>> > appropriate release.
>>>>>>>> >
>>>>>>>> > ==================
>>>>>>>> > But my bug isn't fixed?
>>>>>>>> > ==================
>>>>>>>> >
>>>>>>>> > In order to make timely releases, we will typically
not hold the
>>>>>>>> > release unless the bug in question is a regression from
the
>>>>>>>> previous
>>>>>>>> > release. That being said, if there is something which
is a
>>>>>>>> regression
>>>>>>>> > that has not been correctly targeted please ping me
or a
>>>>>>>> committer to
>>>>>>>> > help target the issue.
>>>>>>>>
>>>>>>>
>>>>>
>>>>> --
>>>>> [image:
>>>>> https://databricks.com/sparkaisummit/north-america?utm_source=email&utm_medium=signature]
>>>>>
>>>>

Mime
View raw message