spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lucas Kacher <lu...@vsco.co>
Subject Re: Build issues with apache-spark-on-k8s.
Date Wed, 28 Mar 2018 13:42:31 GMT
Are you building on the fork or on the official release now? I built v2.3.0
from source w/out issue. One thing I noticed is that I needed to run the
build-image command from the bin which was placed in dist/ as opposed to
the one in the repo (as that's how it copies the necessary targets).

(Failed to reply-all to the list).

On Wed, Mar 28, 2018 at 4:30 AM, Atul Sowani <sowani@gmail.com> wrote:

> Hi,
>
> I built apache-spark-on-k8s from source on Ubuntu 16.04 and it got built
> without errors. Next, I wanted to create docker images, so as explained at
> https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html I
> used sbin/build-push-docker-images.sh to create those. While using this
> script I came across 2 issues:
>
> 1. It references "dockerfiles" directory which should be in "spark",
> however this directory is missing. I created "dockerfiles" directory and
> copied Dockerfiles from resource-managers/kubernetes/docker-minimal-bundle
>
> 2, spark-base dockerfile expects to have some JAR files present in a
> directory called "jars" - this directory is missing. I tried rebuilding the
> code but this directory is not getting generated if it is supposed to be.
>
> My doubt is, if this is a genuine/known issue or am I missing out some
> build steps?
>
> Thanks,
> Atul.
>
>


-- 

*Lucas Kacher*Senior Engineer
-
vsco.co <https://www.vsco.co/>
New York, NY
818.512.5239

Mime
View raw message