spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tal Grynbaum <tal.grynb...@gmail.com>
Subject Re: Please assist: Building Docker image containing spark 2.0
Date Fri, 26 Aug 2016 21:23:54 GMT
Did you specify -Dscala-2.10
As in
./dev/change-scala-version.sh 2.10 ./build/mvn -Pyarn -Phadoop-2.4
-Dscala-2.10 -DskipTests clean package
If you're building with scala 2.10

On Sat, Aug 27, 2016, 00:18 Marco Mistroni <mmistroni@gmail.com> wrote:

> Hello Michael
>     uhm i celebrated too soon
> Compilation of spark on docker image went near the end and then it errored
> out with this message
>
> INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 01:01 h
> [INFO] Finished at: 2016-08-26T21:12:25+00:00
> [INFO] Final Memory: 69M/324M
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal
> net.alchim31.maven:scala-maven-plugin:3.2.2:compile (scala-compile-first)
> on project spark-mllib_2.11: Execution scala-compile-first of goal
> net.alchim31.maven:scala-maven-plugin:3.2.2:compile failed. CompileFailed
> -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the
> -e switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions,
> please read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR]   mvn <goals> -rf :spark-mllib_2.11
> The command '/bin/sh -c ./build/mvn -Pyarn -Phadoop-2.4
> -Dhadoop.version=2.4.0 -DskipTests clean package' returned a non-zero code:
> 1
>
> what am i forgetting?
> once again, last command i launched on the docker file is
>
>
> RUN ./build/mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -DskipTests
> clean package
>
> kr
>
>
>
> On Fri, Aug 26, 2016 at 6:18 PM, Michael Gummelt <mgummelt@mesosphere.io>
> wrote:
>
>> :)
>>
>> On Thu, Aug 25, 2016 at 2:29 PM, Marco Mistroni <mmistroni@gmail.com>
>> wrote:
>>
>>> No i wont accept that :)
>>> I can't believe i have wasted 3 hrs for a space!
>>>
>>> Many thanks MIchael!
>>>
>>> kr
>>>
>>> On Thu, Aug 25, 2016 at 10:01 PM, Michael Gummelt <
>>> mgummelt@mesosphere.io> wrote:
>>>
>>>> You have a space between "build" and "mvn"
>>>>
>>>> On Thu, Aug 25, 2016 at 1:31 PM, Marco Mistroni <mmistroni@gmail.com>
>>>> wrote:
>>>>
>>>>> HI all
>>>>>  sorry for the partially off-topic, i hope there's someone on the list
>>>>> who has tried the same and encountered similar issuse
>>>>>
>>>>> Ok so i have created a Docker file to build an ubuntu container which
>>>>> inlcudes spark 2.0, but somehow when it gets to the point where it has
to
>>>>> kick off  ./build/mvn command, it errors out with the following
>>>>>
>>>>> ---> Running in 8c2aa6d59842
>>>>> /bin/sh: 1: ./build: Permission denied
>>>>> The command '/bin/sh -c ./build mvn -Pyarn -Phadoop-2.4
>>>>> -Dhadoop.version=2.4.0 -DskipTests clean package' returned a non-zero
code:
>>>>> 126
>>>>>
>>>>> I am puzzled as i am root when i build the container, so i should not
>>>>> encounter this issue (btw, if instead of running mvn from the build
>>>>> directory  i use the mvn which i installed on the container, it works
fine
>>>>> but it's  painfully slow)
>>>>>
>>>>> here are the details of my Spark command( scala 2.10, java 1.7 , mvn
>>>>> 3.3.9 and git have already been installed)
>>>>>
>>>>> # Spark
>>>>> RUN echo "Installing Apache spark 2.0"
>>>>> RUN git clone git://github.com/apache/spark.git
>>>>> WORKDIR /spark
>>>>> RUN ./build/mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -DskipTests
>>>>> clean package
>>>>>
>>>>>
>>>>> Could anyone assist pls?
>>>>>
>>>>> kindest regarsd
>>>>>  Marco
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Michael Gummelt
>>>> Software Engineer
>>>> Mesosphere
>>>>
>>>
>>>
>>
>>
>> --
>> Michael Gummelt
>> Software Engineer
>> Mesosphere
>>
>
>

Mime
View raw message