spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hao Ren <inv...@gmail.com>
Subject Unresolved dep when building project with spark 1.6
Date Mon, 29 Feb 2016 10:19:25 GMT
Hi,

I am upgrading my project to spark 1.6.
It seems that the deps are broken.

Deps used in sbt

val scalaVersion = "2.10"
val sparkVersion  = "1.6.0"
val hadoopVersion = "2.7.1"

// Libraries
val scalaTest = "org.scalatest" %% "scalatest" % "2.2.4" % "test"
val sparkSql  = "org.apache.spark" %% "spark-sql" % sparkVersion
val sparkML   = "org.apache.spark" %% "spark-mllib" % sparkVersion
val hadoopAWS = "org.apache.hadoop" % "hadoop-aws" % hadoopVersion
val scopt     = "com.github.scopt" %% "scopt" % "3.3.0"
val jodacvt   = "org.joda" % "joda-convert" % "1.8.1"

Sbt exception:

[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] ::          UNRESOLVED DEPENDENCIES         ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: org.fusesource.leveldbjni#leveldbjni-all;1.8:
org.fusesource.leveldbjni#leveldbjni-all;1.8!leveldbjni-all.pom(pom.original)
origin location must be absolute:
file:/home/invkrh/.m2/repository/org/fusesource/leveldbjni/leveldbjni-all/1.8/leveldbjni-all-1.8.pom
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] Note: Unresolved dependencies path:
[warn] org.fusesource.leveldbjni:leveldbjni-all:1.8
[warn]  +- org.apache.spark:spark-network-shuffle_2.10:1.6.0
[warn]  +- org.apache.spark:spark-core_2.10:1.6.0
[warn]  +- org.apache.spark:spark-catalyst_2.10:1.6.0
[warn]  +- org.apache.spark:spark-sql_2.10:1.6.0
(/home/invkrh/workspace/scala/confucius/botdet/build.sbt#L14)
[warn]  +- org.apache.spark:spark-mllib_2.10:1.6.0
(/home/invkrh/workspace/scala/confucius/botdet/build.sbt#L14)
[warn]  +- fr.leboncoin:botdet_2.10:0.1
sbt.ResolveException: unresolved dependency:
org.fusesource.leveldbjni#leveldbjni-all;1.8:
org.fusesource.leveldbjni#leveldbjni-all;1.8!leveldbjni-all.pom(pom.original)
origin location must be absolute:
file:/home/invkrh/.m2/repository/org/fusesource/leveldbjni/leveldbjni-all/1.8/leveldbjni-all-1.8.pom

Thank you.

-- 
Hao Ren

Data Engineer @ leboncoin

Paris, France

Mime
View raw message