spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yinan Li <liyinan...@gmail.com>
Subject Re: Kubernetes security context when submitting job through k8s servers
Date Mon, 09 Jul 2018 22:55:47 GMT
It's still under design review. It's unlikely that it will go into 2.4.

On Mon, Jul 9, 2018 at 3:46 PM trung kien <kientt86@gmail.com> wrote:

> Thanks Li,
>
> Inread through the ticket, be able to pass pod YAML file would be amazing.
>
> Do you have any target date for production or incubator? I really want to
> try out this feature.
>
> On Mon, Jul 9, 2018 at 4:48 PM Yinan Li <liyinan926@gmail.com> wrote:
>
>> Spark on k8s currently doesn't support specifying a custom
>> SecurityContext of the driver/executor pods. This will be supported by the
>> solution to https://issues.apache.org/jira/browse/SPARK-24434.
>>
>> On Mon, Jul 9, 2018 at 2:06 PM trung kien <kientt86@gmail.com> wrote:
>>
>>> Dear all,
>>>
>>> Is there any way to includes security context (
>>> https://kubernetes.io/docs/tasks/configure-pod-container/security-context/)
>>> when submitting job through k8s servers?
>>>
>>> I'm trying to first spark jobs on Kubernetes through spark-submit:
>>>
>>> bin/spark-submit --master k8s://https://API_SERVERS --deploy-mode
>>> cluster --name spark-pi --class org.apache.spark.examples.SparkPi --conf
>>> spark.kubernetes.namespace=NAMESPACE --conf spark.executor.instances=3
>>> --conf spark.kubernetes.container.image=<SPARK_IMAGE> --conf
>>> spark.kubernetes.driver.pod.name=spark-pi-driver
>>> local:///opt/spark/examples/jars/spark-examples_2.11-2.3.1.jar
>>>
>>> But the job was rejected because the pod (created by spark-submit)
>>> doesn't have security context to run as my account (Our policy doesn't
>>> allow us to runAsUser root)
>>>
>>> I check the code under KubernetesClientApplication.scala
>>> <https://github.com/apache/spark/blob/master/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientApplication.scala>,
>>> it doesn't seems to support security context by configuration.
>>>
>>> Is there any solution to get arround this issue? is there any patch that
>>> support this?
>>>
>>> --
>>> Thanks
>>> Kien
>>>
>> --
> Thanks
> Kien
>

Mime
View raw message