sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devin G. Bost (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-3397) Java version issues with Sqoop on AWS EMR
Date Fri, 19 Oct 2018 23:32:00 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-3397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16657555#comment-16657555
] 

Devin G. Bost commented on SQOOP-3397:
--------------------------------------

Turns out that the solution was simpler than I expected. Just ensure that both Java versions
are installed on the machine (e.g. install Java 7 via: sudo yum install java-1.7.0 java-1.7.0-openjdk-devel
-y ) and then put:

{{ export JAVA_HOME=/usr/lib/jvm/java-1.7.0}}

into:

{{ /usr/lib/sqoop/conf/sqoop-env.sh}}

and ensure that your defaults are set back to Java 8 for everything else, like:

{{NR_OF_JRE_OPTIONS=$(echo 0 | alternatives --config java 2>/dev/null | grep 'There ' |
awk '\{print $3}' | tail -1)}}
{{NR_OF_SDK_OPTIONS=$(echo 0 | alternatives --config javac 2>/dev/null | grep 'There '
| awk '\{print $3}' | tail -1)}}{{echo $(($NR_OF_SDK_OPTIONS - 1)) | sudo alternatives --config
javac}}
{{echo $(($NR_OF_JRE_OPTIONS - 1)) | sudo alternatives --config java}}

(Modified from here: [https://gist.github.com/pstorch/c217d8324c4133a003c4] )

 

 

> Java version issues with Sqoop on AWS EMR
> -----------------------------------------
>
>                 Key: SQOOP-3397
>                 URL: https://issues.apache.org/jira/browse/SQOOP-3397
>             Project: Sqoop
>          Issue Type: Bug
>    Affects Versions: 1.4.7
>         Environment: Amazon EMR emr-5.14.0 and up.
>            Reporter: Devin G. Bost
>            Priority: Major
>
> The current build of Sqoop 1.4.7 appears to require Java 7. 
> The version of Sqoop 1.4.7 on Amazon's Elastic Map Reduce (EMR) platform appears to be
running on Java 8. 
> We need a way to be able to build Sqoop extensions on Java 8 so that they are compatible
with the version of Java running on the EMR versions of Sqoop. Otherwise, when attempts to
run the extension of Sqoop are made (such as for a custom Manager class), Java version mismatch
errors occur that block compilation of the class files that need to be generated for imports. 
> How can we build Sqoop 1.4.7 on Java 8 so that extensions and plugins can be built with
Java 8 so that they will still be compatible with the architecture of the other Apache tools
(such as Hadoop) that otherwise break on EMR instances when the Java version must be switched
to Java 7 to support Sqoop extensions and plugins?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message