spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <so...@cloudera.com>
Subject Re: Anyone know how to bypass tools.jar problem in JDK9 when mvn clean install SPARK code
Date Thu, 21 Dec 2017 22:56:11 GMT
You need to run ./dev/change-scala-version.sh 2.12 first

On Thu, Dec 21, 2017 at 4:38 PM Zhang, Liyun <liyun.zhang@intel.com> wrote:

> Hi all:
>
>   Now I am using JDK9 to compile Spark by (mvn clean install –DskipTests),
> but exception was thrown
>
>
>
> [root@bdpe41 spark_source]# java -version
>
> java version "9.0.1"
>
> Java(TM) SE Runtime Environment (build 9.0.1+11)
>
> Java HotSpot(TM) 64-Bit Server VM (build 9.0.1+11, mixed mode)
>
>
>
> #mvn clean install -Pscala-2.12  -Pyarn -Pparquet-provided -DskipTests
> -X>log.mvn.clean.installed 2>&1
>
>
>
> 654189 [INFO] Spark Project Hive ................................. SUCCESS
> [ 30.708 s]
>
> 654190 [INFO] Spark Project REPL ................................. SUCCESS
> [  2.795 s]
>
> 654191 [INFO] Spark Project YARN Shuffle Service ................. SUCCESS
> [  6.411 s]
>
> 654192 [INFO] Spark Project YARN ................................. FAILURE
> [  0.047 s]
>
> 654193 [INFO] Spark Project Assembly ............................. SKIPPED
>
> 654194 [INFO] Spark Integration for Kafka 0.10 ................... SKIPPED
>
> 654195 [INFO] Kafka 0.10 Source for Structured Streaming ......... SKIPPED
>
> 654196 [INFO] Spark Project Examples ............................. SKIPPED
>
> 654197 [INFO] Spark Integration for Kafka 0.10 Assembly .......... SKIPPED
>
> 654198 [INFO]
> ------------------------------------------------------------------------
>
> 654199 [INFO] BUILD FAILURE
>
> 654200 [INFO]
> ------------------------------------------------------------------------
>
> 654201 [INFO] Total time: 07:04 min
>
> 654202 [INFO] Finished at: 2017-12-21T03:38:40+08:00
>
> 654203 [INFO] Final Memory: 85M/280M
>
> 654204 [INFO]
> ------------------------------------------------------------------------
>
> 654205 [ERROR] Failed to execute goal on project spark-yarn_2.12: Could
> not resolve dependencies for project
> org.apache.spark:spark-yarn_2.12:jar:2.3.0-SNAPS       HOT: Could not find
> artifact jdk.tools:jdk.tools:jar:1.6 at specified path
> /home/zly/prj/oss/jdk-9.0.1/../lib/tools.jar -> [Help 1]
>
> 654206 org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
> execute goal on project spark-yarn_2.12: Could not resolve dependencies for
> project        org.apache.spark:spark-yarn_2.12:jar:2.3.0-SNAPSHOT: Could
> not find artifact jdk.tools:jdk.tools:jar:1.6 at specified path
> /home/zly/prj/oss/jdk-9.0.1       /../lib/tools.jar
>
>
>
>
>
> There is no tools.jar in JDK9
> <http://blog.codefx.org/java/dev/how-java-9-and-project-jigsaw-may-break-your-code/#Internal-JARs-Become-Unavailable>.
> I need to generate spark 2.3-SNAPSHOT in my local mvn repository to build
> other component(Hive on Spark) .  Anyone knows how to bypass this problem?
>
>
>
>
>
>
>
>
>
> Best Regards
>
> Kelly Zhang/Zhang,Liyun
>
>
>

Mime
View raw message