spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Antoine DUBOIS <antoine.dub...@cc.in2p3.fr>
Subject Using spark and mesos container with host_path volume
Date Mon, 03 Dec 2018 15:44:29 GMT
Hello, 
I'm trying to mount a local ceph volume to my mesos container. 
My cephfs is mounted on all agent at /ceph 
I'm using spark 2.4 with hadoop 3.11 and I'm not using Docker to deploy spark. 
The only option I could find to mount a volume though is the following (which is also a line
I added to spark-defaults.conf in both my client and in the .tgz deployed on mesos): 

spark.mesos.executor.docker.volumes /ceph:/mnt:rw 

However it seems that the volume is never mounted on mesos containers and using spark-shell
on my mesos cluster I'm unable to access any thing in /mnt. 
I have activated the volume/host_path on all mesos agent and I don't really know what could
I do now. 

Is there a way to do what I want with mesos containizer or do I have to use docker (a without
docker solution would be so lovely). 

Thank you in advance for any hint or help. 

Antoine



Mime
View raw message