spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sushil.chaudhary" <sushil.chaudh...@capitalone.com>
Subject Spark 2.3.1 leaves _temporary dir back on s3 even after write to s3 is done.
Date Thu, 04 Oct 2018 20:01:41 GMT
folks,
We recently upgraded to 2.3.1 and we started seeing that, the spark jobs
leaves _temporary directory in the s3 even though write to s3 already
finish. It do not cleanup the temporary directory. 
Hadoop version 2.8.  is there a way to control it?



--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message