spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Imran Rajjad <raj...@gmail.com>
Subject Re: No space left on device
Date Wed, 18 Oct 2017 05:07:10 GMT
did you try calling rdd.unpersist()

On Wed, Oct 18, 2017 at 10:04 AM, Mina Aslani <aslanimina@gmail.com> wrote:

> Hi,
>
> I get "No space left on device" error in my spark worker:
>
> Error writing stream to file /usr/spark-2.2.0/work/app-......./0/stderr
> java.io.IOException: No space left on device
>
> In my spark cluster, I have one worker and one master.
> My program consumes stream of data from kafka and publishes the result
> into kafka. I set my RDD = null after I finish working, so that
> intermediate shuffle files are removed quickly.
>
> How can I avoid "No space left on device"?
>
> Best regards,
> Mina
>



-- 
I.R

Mime
View raw message