I had the same problem, and tried to compile with mvn -Pnetlib-lgpl

$ mvn -Pnetlib-lgpl -Pyarn -Phadoop-2.3 -Dhadoop.version=2.3.0 -DskipTests clean package

Unfortunately, the resulting assembly jar still lacked the netlib-system class. This command :

$ jar tvf assembly/target/scala-2.10/spark-assembly-1.1.1-hadoop2.3.0.jar |grep netlib | grep Native

returns nothing...

(and for some reason, including the netlib-all in my shipped jar did not solve the problem either, apparently the classloader does not find the class)

In Spark, the profile is defined in mllib submodule, but the -Pnetlib-lgpl seems not to be transmitted to the child from the parent pom.xml

I don't know how to fix that cleanly (I just added <activeByDefault>true</activeByDefault> in mllib's pom.xml), maybe it's just a problem with my maven version (3.0.5)


I tried building Spark from the source, by downloading it and running:

mvn -Pnetlib-lgpl -DskipTests clean package

Guillaume PITEL, Président
+33(0)626 222 431

eXenSa S.A.S.
41, rue Périer - 92120 Montrouge - FRANCE
Tel +33(0)184 163 677 / Fax +33(0)972 283 705