spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicholas Chammas <>
Subject Re: Use Hadoop-3.2 as a default Hadoop profile in 3.0.0?
Date Sun, 03 Nov 2019 17:32:12 GMT
On Fri, Nov 1, 2019 at 8:41 AM Steve Loughran <>

> It would be really good if the spark distributions shipped with later
> versions of the hadoop artifacts.

I second this. If we need to keep a Hadoop 2.x profile around, why not make
it Hadoop 2.8 or something newer?

Koert Kuipers <> wrote:

> given that latest hdp 2.x is still hadoop 2.7 bumping hadoop 2 profile to
> latest would probably be an issue for us.

When was the last time HDP 2.x bumped their minor version of Hadoop? Do we
want to wait for them to bump to Hadoop 2.8 before we do the same?

View raw message