spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <sro...@gmail.com>
Subject Re: Packages to release in 3.0.0-preview
Date Sun, 27 Oct 2019 14:24:45 GMT
Is the Spark artifact actually any different between those builds? I
thought it just affected what else was included in the binary tarball.
If it matters, yes I'd publish a "Hadoop 3" version to Maven. (Scala
2.12 is the only supported Scala version).

On Sun, Oct 27, 2019 at 4:35 AM Yuming Wang <wgyumg@gmail.com> wrote:
>
> Do we need to publish the Scala 2.12 + hadoop 3.2 jar packages to the Maven repository?
Otherwise it will throw a NoSuchMethodError on Java 11.
> Here is an example:
> https://github.com/wangyum/test-spark-jdk11/blob/master/src/test/scala/test/spark/HiveTableSuite.scala#L34-L38
> https://github.com/wangyum/test-spark-jdk11/commit/927ce7d3766881fba98f2434055fa3a1d1544ad2/checks?check_suite_id=283076578
>
>
> On Sat, Oct 26, 2019 at 10:41 AM Takeshi Yamamuro <linguin.m.s@gmail.com> wrote:
>>
>> Thanks for that work!
>>
>> > I don't think JDK 11 is a separate release (by design). We build
>> > everything targeting JDK 8 and it should work on JDK 11 too.
>> +1. a single package working on both jvms looks nice.
>>
>>
>> On Sat, Oct 26, 2019 at 4:18 AM Sean Owen <srowen@gmail.com> wrote:
>>>
>>> I don't think JDK 11 is a separate release (by design). We build
>>> everything targeting JDK 8 and it should work on JDK 11 too.
>>>
>>> So, just two releases, but, frankly I think we soon need to stop
>>> multiple releases for multiple Hadoop versions, and stick to Hadoop 3.
>>> I think it's fine to try to release for Hadoop 2 as the support still
>>> exists, and because the difference happens to be larger due to the
>>> different Hive dependency.
>>>
>>> On Fri, Oct 25, 2019 at 2:08 PM Xingbo Jiang <jiangxb1987@gmail.com> wrote:
>>> >
>>> > Hi all,
>>> >
>>> > I would like to bring out a discussion on how many packages shall be released
in 3.0.0-preview, the ones I can think of now:
>>> >
>>> > * scala 2.12 + hadoop 2.7
>>> > * scala 2.12 + hadoop 3.2
>>> > * scala 2.12 + hadoop 3.2 + JDK 11
>>> >
>>> > Do you have other combinations to add to the above list?
>>> >
>>> > Cheers,
>>> >
>>> > Xingbo
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>>
>>
>>
>> --
>> ---
>> Takeshi Yamamuro

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org


Mime
View raw message