spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Holden Karau <hol...@pigscanfly.ca>
Subject Re: [Spark on Google Kubernetes Engine] Properties File Error
Date Mon, 30 Apr 2018 19:00:21 GMT
So, while its not perfect, I have a guide focused on running custom Spark
on GKE
https://cloud.google.com/blog/big-data/2018/03/testing-future-apache-spark-releases-and-changes-on-google-kubernetes-engine-and-cloud-dataproc
and
if you want to run pre-built Spark on GKE there is a solutions article -
https://cloud.google.com/solutions/spark-on-kubernetes-engine which could
be relevant.

On Mon, Apr 30, 2018 at 7:51 PM, Eric Wang <ericswang1996@gmail.com> wrote:

> Hello all,
>
> I've been trying to spark-submit a job to the Google Kubernetes Engine but
> I keep encountering a "Exception in thread "main"
> java.lang.IllegalArgumentException: Server properties file given at
> /opt/spark/work-dir/driver does not exist or is not a file."
> error. I'm unsure of how to even begin debugging this so any help would be
> greatly appreciated. I've attached the logs and the full spark-submit
> command I'm running here: https://gist.github.com/
> erkkel/c04a0b5ca60ad755cf62e9ad18e5b7ed
>
> For reference, I've been following this guide: https://apache-spark-
> on-k8s.github.io/userdocs/running-on-kubernetes.html
>
> Thanks,
> Eric
>
>


-- 
Twitter: https://twitter.com/holdenkarau

Mime
View raw message