spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cheney Sun <>
Subject too many temporary app files left after app finished
Date Tue, 27 May 2014 09:22:05 GMT

We use spark 0.9.1 in standalone mode.
We found lots of app temporary files didn't get removed in each worker
local file system even while the job was finished. These folder have names
such as "app-20140516120842-0203".

These files occupied so many disk storage that we have to run a deamon
script to remove them periodically. I thinks this method is ugly.
Did anybody run into this issue as well? Is there any conf to delete the
app temporary files automatically once job finished?


View raw message