spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yinan Li <liyinan...@gmail.com>
Subject Re: [Spark on Google Kubernetes Engine] Properties File Error
Date Mon, 30 Apr 2018 19:09:48 GMT
 Which version of Spark are you using to run spark-submit, and which
version of Spark your container image is based off? This looks to be caused
my mismatched versions of Spark used for spark-submit and for the
driver/executor at runtime.

On Mon, Apr 30, 2018 at 12:00 PM, Holden Karau <holden@pigscanfly.ca> wrote:

> So, while its not perfect, I have a guide focused on running custom Spark
> on GKE https://cloud.google.com/blog/big-data/2018/03/testing-
> future-apache-spark-releases-and-changes-on-google-
> kubernetes-engine-and-cloud-dataproc and if you want to run pre-built
> Spark on GKE there is a solutions article - https://cloud.google.com/
> solutions/spark-on-kubernetes-engine which could be relevant.
>
> On Mon, Apr 30, 2018 at 7:51 PM, Eric Wang <ericswang1996@gmail.com>
> wrote:
>
>> Hello all,
>>
>> I've been trying to spark-submit a job to the Google Kubernetes Engine
>> but I keep encountering a "Exception in thread "main"
>> java.lang.IllegalArgumentException: Server properties file given at
>> /opt/spark/work-dir/driver does not exist or is not a file."
>> error. I'm unsure of how to even begin debugging this so any help would
>> be greatly appreciated. I've attached the logs and the full spark-submit
>> command I'm running here: https://gist.github.com/
>> erkkel/c04a0b5ca60ad755cf62e9ad18e5b7ed
>>
>> For reference, I've been following this guide: https://apache-spark-on
>> -k8s.github.io/userdocs/running-on-kubernetes.html
>>
>> Thanks,
>> Eric
>>
>>
>
>
> --
> Twitter: https://twitter.com/holdenkarau
>

Mime
View raw message