spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chetan Khatri <chetan.opensou...@gmail.com>
Subject Re: No space left on device
Date Wed, 18 Oct 2017 05:06:38 GMT
Process data in micro batch
On 18-Oct-2017 10:36 AM, "Chetan Khatri" <chetan.opensource@gmail.com>
wrote:

> Your hard drive don't have much space
> On 18-Oct-2017 10:35 AM, "Mina Aslani" <aslanimina@gmail.com> wrote:
>
>> Hi,
>>
>> I get "No space left on device" error in my spark worker:
>>
>> Error writing stream to file /usr/spark-2.2.0/work/app-......./0/stderr
>> java.io.IOException: No space left on device
>>
>> In my spark cluster, I have one worker and one master.
>> My program consumes stream of data from kafka and publishes the result
>> into kafka. I set my RDD = null after I finish working, so that
>> intermediate shuffle files are removed quickly.
>>
>> How can I avoid "No space left on device"?
>>
>> Best regards,
>> Mina
>>
>

Mime
View raw message