spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Martin Loncaric (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-27014) Support removal of jars and Spark binaries from Mesos driver and executor sandboxes
Date Thu, 28 Feb 2019 20:22:00 GMT
Martin Loncaric created SPARK-27014:
---------------------------------------

             Summary: Support removal of jars and Spark binaries from Mesos driver and executor
sandboxes
                 Key: SPARK-27014
                 URL: https://issues.apache.org/jira/browse/SPARK-27014
             Project: Spark
          Issue Type: New Feature
          Components: Mesos
    Affects Versions: 2.4.0
            Reporter: Martin Loncaric


Currently, each Spark application run on Mesos leaves behind at least 500MB of data in sandbox
directories, coming from Spark binaries and copied URIs. These can build up as a disk leak,
causing major issues on Mesos clusters unless their grace period for sandbox directories is
very short.

Spark should have a feature to delete these (from both driver and executor sandboxes) on teardown.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message