hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinayakumar B (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-10115) Exclude duplicate jars in hadoop package under different component's lib
Date Tue, 10 Mar 2015 03:30:41 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-10115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Vinayakumar B updated HADOOP-10115:
    Attachment: HADOOP-10115-007.patch

Updated the patch.
bq. Could we use a maven variable for this instead of cd/pwd?
Yes, done. Used as {code}ROOT=$(cd "${project.build.directory}"/../..;pwd){code}

bq. Could you add a comment here that it's important we process the hadoop-common project
first, so that common always has all the dependencies it declares?
bq. Should the yarn get processed before the NFS projects?
NFS projects are depend on common and hdfs only respectively.
And they will be copied to common/hdfs directory itself. So copying these will not affect
much for the Yarn projects.

> Exclude duplicate jars in hadoop package under different component's lib
> ------------------------------------------------------------------------
>                 Key: HADOOP-10115
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10115
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: build
>    Affects Versions: 3.0.0, 2.2.0
>            Reporter: Vinayakumar B
>            Assignee: Vinayakumar B
>              Labels: common, hdfs, mapreduce, nfs, yarn
>         Attachments: HADOOP-10115-004.patch, HADOOP-10115-005.patch, HADOOP-10115-006.patch,
HADOOP-10115-007.patch, HADOOP-10115.patch, HADOOP-10115.patch, HADOOP-10115.patch
> In the hadoop package distribution there are more than 90% of the jars are duplicated
in multiple places.
> For Ex:
> almost all jars in share/hadoop/hdfs/lib are already there in share/hadoop/common/lib
> Same case for all other lib in share directory.
> Anyway for all the daemon processes all directories are added to classpath.
> So to reduce the package distribution size and the classpath overhead, remove the duplicate
jars from the distribution.

This message was sent by Atlassian JIRA

View raw message