falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Srikanth Sundarrajan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-106) Falcon compilation fails for hadoop-2 profile
Date Sat, 14 Sep 2013 19:37:51 GMT

    [ https://issues.apache.org/jira/browse/FALCON-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767552#comment-13767552

Srikanth Sundarrajan commented on FALCON-106:

If I understand the intention of this JIRA correctly, we want to be able to 

1. Provide a profile name for compilation to compile against a specific dependency of hadoop,
as opposed to a system property and auto profile activation based on that
2. Have a default profile so that users dont have to always provide a profile name for compilation
3. System builds correctly and unit/integration tests pass for (no profile switch (default
hadoop-1), hadoop-1, hadoop-2, distributed (default hadoop-1), distributed/hadoop-1, distributed/hadoop-2)
4. Not include hadoop artifacts for a specific module, if it is genuinely not required.

If we relax the requirement to always pass the hadoop profile name when building for distributed,
the problem becomes very trivial. I would assume that it is not too limiting and is a reasonable
state to be in.

# Simply get rid of profiles section in each of the child modules
# Keep the profiles to the parent pom and profiles to include appropriate dependency management
& dependencies sections with scope which can be extended per module
# Each module gets to choose what is the acceptable dependency scope for the hadoop dependency

This is more or less similar to what has already be proposed as a patch in https://issues.apache.org/jira/secure/attachment/12602545/falcon-build-2.x.v2.patch
by [~sureshms] & further commented by [~shwethags] in https://issues.apache.org/jira/browse/FALCON-106?focusedCommentId=13760169&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13760169

Also the fact that it takes a while to build falcon each time, seems to make it difficult
to iterate over and fix issues. I will open a JIRA to disable checkstyle, findbugs, docs generation,
sources packaging by default and have them run only during test-patch. Also will create another
one to revisit test cases involving MiniDFSCluster and see if there genuinely is a need for
these and exclude some integration tests in the normal build and include them only for test-patch
and nightly builds by jenkins.
> Falcon compilation fails for hadoop-2 profile
> ---------------------------------------------
>                 Key: FALCON-106
>                 URL: https://issues.apache.org/jira/browse/FALCON-106
>             Project: Falcon
>          Issue Type: Bug
>          Components: build-tools
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>         Attachments: falcon-build-2.x.patch, falcon-build-2.x.v2.patch, falcon-build-2.x.v2.patch,
> Mainly due to hadoop-core and hadoop-test artifacts not being available in hadoop 2.x.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message