spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcelo Vanzin <>
Subject Re: Do we need to finally update Guava?
Date Mon, 16 Dec 2019 17:36:35 GMT
Great that Hadoop has done it (which, btw, probably means that Spark
won't work with that version of Hadoop yet), but Hive also depends on
Guava, and last time I tried, even Hive 3.x did not work with Guava

(Newer Hadoop versions also have a new artifact that shades a lot of
dependencies, which would be great for Spark. But since Spark uses
some test artifacts from Hadoop, that may be a bit tricky, since I
don't believe those are shaded.)

On Sun, Dec 15, 2019 at 8:08 AM Sean Owen <> wrote:
> See for example:
> This is a dicey dependency that we have been reluctant to update as a)
> Hadoop used an old version and b) Guava versions are incompatible
> after a few releases.
> But Hadoop is going all the way from 11 to 27 in Hadoop 3.2.1. Time to
> match that? I haven't assessed how much internal change it requires.
> If it's a lot, well, that makes it hard, as we need to stay compatible
> with Hadoop 2 / Guava 11-14. But then that causes a problem updating
> past Hadoop 3.2.0.
> ---------------------------------------------------------------------
> To unsubscribe e-mail:


To unsubscribe e-mail:

View raw message