spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mildebrandt <>
Subject Problems building Spark for Hadoop 1.0.3
Date Mon, 13 Oct 2014 21:39:48 GMT

After reading the following pages:

I came up with this build command:
mvn -Dhadoop.version=1.0.3 -DskipTests -Pdeb -Phadoop-1.0.3 -Phive
-Phadoop-provided clean package

This fails with the following error:
[WARNING] The requested profile "hadoop-1.0.3" could not be activated
because it does not exist.
[ERROR] Failed to execute goal on project spark-parent: Could not resolve
dependencies for project org.apache.spark:spark-parent:pom:1.1.0: The
following artifacts could not be resolved:
org.apache.hadoop:hadoop-yarn-client:jar:1.0.3: Failure to find
org.apache.hadoop:hadoop-yarn-api:jar:1.0.3 in was cached in the local repository,
resolution will not be reattempted until the update interval of central has
elapsed or updates are forced -> [Help 1]

If I remove the hadoop-provided profile, it works fine. So my questions:
 - Why isn't there a hadoop-1.0.3 profile? The
hadoop-third-party-distributions.html page seems to suggest that it would
 - It's true, I don't see a hadoop-yarn-api:jar:1.0.3 in the maven
repository. I'd like to build without yarn support. And ideas why it's
pulling yarn? Is it due to the fact that the hadoop-1.0.3 profile doesn't
 - Since running without the hadoop-provided profile works, is there any
issue running that version of Spark against my Apache 1.0.3 cluster?


View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message