hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsuyoshi Ozawa (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14284) Shade Guava everywhere
Date Fri, 05 May 2017 22:00:06 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15999022#comment-15999022
] 

Tsuyoshi Ozawa commented on HADOOP-14284:
-----------------------------------------

[~stack] thanks you for your help.

{quote}
Tell us more please. Shading bundles the relocated .class files of guava and curator; they
are included in the thirdparty jar... and the thirdparty jar is on the classpath, no?
{quote}

Yes, they are. This is a list of the contents of hadoop-shaded-thirdparty: https://gist.github.com/oza/62fdea66a55c86eda02d2a8530058153
The list shows that shading works well. As a result, mvn install -DskipTests succeeds with
the patch: it means the thirdparty jar is on the classpath at compile time.

On the other hand, mvn test of hadoop-auth fails with the patch although I added dependency
on hadoop-shaded-thirdparty in compile scope.

https://github.com/oza/hadoop/blob/HADOOP-14284/hadoop-common-project/hadoop-auth/pom.xml#L46

the result of mvn test is as follows:
{quote}
Running org.apache.hadoop.security.authentication.client.TestKerberosAuthenticator
Tests run: 20, Failures: 5, Errors: 11, Skipped: 0, Time elapsed: 23.407 sec <<<
FAILURE! - in org.apache.hadoop.security.authentication.client.TestKerberosAuthenticator
testNotAuthenticatedWithMultiAuthHandler\[0\](org.apache.hadoop.security.authentication.client.TestKerberosAuthenticator)
 Time elapsed: 1.649 sec  <<< ERROR!
java.lang.NoClassDefFoundError: org/apache/hadoop/shaded/com/google/common/base/Splitter
{quote}

I thought that this might be a problem because the thirdparty jar is not on the classpath
at testing time,  Please let me know if I'm missing something.

> Shade Guava everywhere
> ----------------------
>
>                 Key: HADOOP-14284
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14284
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: build
>    Affects Versions: 3.0.0-alpha3
>            Reporter: Andrew Wang
>            Assignee: Tsuyoshi Ozawa
>            Priority: Blocker
>         Attachments: HADOOP-14238.pre001.patch, HADOOP-14284.002.patch, HADOOP-14284.004.patch,
HADOOP-14284.007.patch, HADOOP-14284.010.patch
>
>
> HADOOP-10101 upgraded the guava version for 3.x to 21.
> Guava is broadly used by Java projects that consume our artifacts. Unfortunately, these
projects also consume our private artifacts like {{hadoop-hdfs}}. They also are unlikely on
the new shaded client introduced by HADOOP-11804, currently only available in 3.0.0-alpha2.
> We should shade Guava everywhere to proactively avoid breaking downstreams. This isn't
a requirement for all dependency upgrades, but it's necessary for known-bad dependencies like
Guava.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message