falcon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Srikanth Sundarrajan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FALCON-106) Falcon compilation fails for hadoop-2 profile
Date Sat, 14 Sep 2013 19:37:51 GMT

    [ https://issues.apache.org/jira/browse/FALCON-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13767552#comment-13767552
] 

Srikanth Sundarrajan commented on FALCON-106:
---------------------------------------------

If I understand the intention of this JIRA correctly, we want to be able to 

1. Provide a profile name for compilation to compile against a specific dependency of hadoop,
as opposed to a system property and auto profile activation based on that
2. Have a default profile so that users dont have to always provide a profile name for compilation
3. System builds correctly and unit/integration tests pass for (no profile switch (default
hadoop-1), hadoop-1, hadoop-2, distributed (default hadoop-1), distributed/hadoop-1, distributed/hadoop-2)
4. Not include hadoop artifacts for a specific module, if it is genuinely not required.

If we relax the requirement to always pass the hadoop profile name when building for distributed,
the problem becomes very trivial. I would assume that it is not too limiting and is a reasonable
state to be in.

# Simply get rid of profiles section in each of the child modules
# Keep the profiles to the parent pom and profiles to include appropriate dependency management
& dependencies sections with scope which can be extended per module
# Each module gets to choose what is the acceptable dependency scope for the hadoop dependency

This is more or less similar to what has already be proposed as a patch in https://issues.apache.org/jira/secure/attachment/12602545/falcon-build-2.x.v2.patch
by [~sureshms] & further commented by [~shwethags] in https://issues.apache.org/jira/browse/FALCON-106?focusedCommentId=13760169&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13760169

Also the fact that it takes a while to build falcon each time, seems to make it difficult
to iterate over and fix issues. I will open a JIRA to disable checkstyle, findbugs, docs generation,
sources packaging by default and have them run only during test-patch. Also will create another
one to revisit test cases involving MiniDFSCluster and see if there genuinely is a need for
these and exclude some integration tests in the normal build and include them only for test-patch
and nightly builds by jenkins.
                
> Falcon compilation fails for hadoop-2 profile
> ---------------------------------------------
>
>                 Key: FALCON-106
>                 URL: https://issues.apache.org/jira/browse/FALCON-106
>             Project: Falcon
>          Issue Type: Bug
>          Components: build-tools
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>         Attachments: falcon-build-2.x.patch, falcon-build-2.x.v2.patch, falcon-build-2.x.v2.patch,
falcon-build-2.x.v3.patch
>
>
> Mainly due to hadoop-core and hadoop-test artifacts not being available in hadoop 2.x.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message