spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephen Boesch <java...@gmail.com>
Subject Current way to include hive in a build
Date Thu, 17 Jul 2014 23:00:26 GMT
Having looked at trunk make-distribution.sh the --with-hive and --with-yarn
are now deprecated.

Here is the way I have built it:

Added to pom.xml:

   <profile>
      <id>cdh5</id>
      <activation>
        <activeByDefault>false</activeByDefault>
      </activation>
      <properties>
        <hadoop.version>2.3.0-cdh5.0.0</hadoop.version>
        <yarn.version>2.3.0-cdh5.0.0</yarn.version>
        <hbase.version>0.96.1.1-cdh5.0.0</hbase.version>
        <zookeeper.version>3.4.5-cdh5.0.0</zookeeper.version>
      </properties>
    </profile>

*mvn -Pyarn -Pcdh5 -Phive -Dhadoop.version=2.3.0-cdh5.0.1
-Dyarn.version=2.3.0-cdh5.0.0 -DskipTests clean package*


[INFO]
------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Spark Project Parent POM .......................... SUCCESS [3.165s]
[INFO] Spark Project Core ................................ SUCCESS
[2:39.504s]
[INFO] Spark Project Bagel ............................... SUCCESS [7.596s]
[INFO] Spark Project GraphX .............................. SUCCESS [22.027s]
[INFO] Spark Project ML Library .......................... SUCCESS [36.284s]
[INFO] Spark Project Streaming ........................... SUCCESS [24.309s]
[INFO] Spark Project Tools ............................... SUCCESS [3.147s]
[INFO] Spark Project Catalyst ............................ SUCCESS [20.148s]
[INFO] Spark Project SQL ................................. SUCCESS [18.560s]
*[INFO] Spark Project Hive ................................ FAILURE
[33.962s]*

[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-dependency-plugin:2.4:copy-dependencies
(copy-dependencies) on project spark-hive_2.10: Execution copy-dependencies
of goal
org.apache.maven.plugins:maven-dependency-plugin:2.4:copy-dependencies
failed: Plugin org.apache.maven.plugins:maven-dependency-plugin:2.4 or one
of its dependencies could not be resolved: Could not find artifact
commons-logging:commons-logging:jar:1.0.4 -> [Help 1]

Anyone who is presently building with -Phive and has a suggestion for this?

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message