spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marius Soutier <mps....@gmail.com>
Subject Re: Snappy temp files not cleaned up
Date Thu, 06 Nov 2014 09:46:11 GMT
Default value is infinite, so you need to enable it. Personally I’ve setup a couple of cron
jobs to clean up /tmp and /var/run/spark.

On 06.11.2014, at 08:15, Romi Kuntsman <romi@totango.com> wrote:

> Hello,
> 
> Spark has an internal cleanup mechanism
> (defined by spark.cleaner.ttl, see http://spark.apache.org/docs/latest/configuration.html)
> which cleans up tasks and stages.
> 
> However, in our installation, we noticed that Snappy temporary files and never cleaned
up.
> 
> Is it a misconfiguration? Missing feature? How do you deal with build-up of temp files?
> 
> Thanks,
> 
> Romi Kuntsman, Big Data Engineer
> http://www.totango.com


Mime
View raw message