spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: maven doesn't build dependencies with Scala 2.11
Date Sun, 18 Jan 2015 02:46:26 GMT
There're 3 jars under lib_managed/jars directory with and
without -Dscala-2.11 flag.

Difference between scala-2.10 and scala-2.11 profiles is that scala-2.10
profile has the following:
      <modules>
        <module>external/kafka</module>
      </modules>

FYI

On Sat, Jan 17, 2015 at 4:07 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> I did the following:
>  1655  dev/change-version-to-2.11.sh
>  1657  mvn -DHADOOP_PROFILE=hadoop-2.4 -Pyarn,hive -Phadoop-2.4
> -Dscala-2.11 -DskipTests clean package
>
> And mvn command passed.
>
> Did you see any cross-compilation errors ?
>
> Cheers
>
> BTW the two links you mentioned are consistent in terms of building for
> Scala 2.11
>
> On Sat, Jan 17, 2015 at 3:43 PM, Walrus theCat <walrusthecat@gmail.com>
> wrote:
>
>> Hi,
>>
>> When I run this:
>>
>> dev/change-version-to-2.11.sh
>> mvn -Pyarn -Phadoop-2.4 -Dscala-2.11 -DskipTests clean package
>>
>> as per here
>> <https://spark.apache.org/docs/latest/building-spark.html#building-for-scala-211>,
>> maven doesn't build Spark's dependencies.
>>
>> Only when I run:
>>
>> dev/change-version-to-2.11.sh
>> sbt/sbt -Pyarn -Phadoop-2.4 -Dscala-2.11 -DskipTests  clean package
>>
>> as gathered from here <https://github.com/ScrapCodes/spark-1/blob/patch-3/docs/building-spark.md>,
do I get Spark's dependencies built without any cross-compilation errors.
>>
>> *Question*:
>>
>> - How can I make maven do this?
>>
>> - How can I specify the use of Scala 2.11 in my own .pom files?
>>
>> Thanks
>>
>>
>

Mime
View raw message