spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thodoris Zois <z...@ics.forth.gr>
Subject Re: Scala program to spark-submit on k8 cluster
Date Fri, 06 Apr 2018 14:53:46 GMT
If you are looking for a Spark scheduler that runs on top of Kubernetes then this is the way
to go: https://github.com/apache/spark/blob/master/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala

You can also have a look here: 
https://apache-spark-on-k8s.github.io/userdocs/running-on-kubernetes.html

- Thodoris

> On 6 Apr 2018, at 15:04, Kittu M <kittum0723@gmail.com> wrote:
> 
> Yes Yinan I’m looking for a Scala program which submits a Spark job to a k8s cluster
by running spark-submit programmatically 
> 
>> On Wednesday, April 4, 2018, Yinan Li <liyinan926@gmail.com> wrote:
>> Hi Kittu,
>> 
>> What do you mean by "a Scala program"? Do you mean a program that submits a Spark
job to a k8s cluster by running spark-submit programmatically, or some example Scala application
that is to run on the cluster? 
>> 
>>> On Wed, Apr 4, 2018 at 4:45 AM, Kittu M <kittum0723@gmail.com> wrote:
>>> Hi,
>>> 
>>> I’m looking for a Scala program to spark submit a Scala application (spark
2.3 job) on k8 cluster .
>>> 
>>> Any help  would be much appreciated. Thanks 
>>> 
>>> 
>> 

Mime
View raw message