spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rishi Shah <rishishah.s...@gmail.com>
Subject Re: [pyspark2.4+] A lot of tasks failed, but job eventually completes
Date Mon, 06 Jan 2020 04:19:43 GMT
Thanks Hemant, underlying data volume increased from 550GB to 690GB and now
the same job doesn't succeed. I tried incrementing executor memory to 20G
as well, still fails. I am running this in Databricks and start cluster
with 20G assigned to spark.executor.memory property.

Also some more information on the job, I have about 4 window functions on
this dataset before it gets written out.

Any other ideas?

Thanks,
-Shraddha

On Sun, Jan 5, 2020 at 11:06 PM hemant singh <hemant2184@gmail.com> wrote:

> You can try increasing the executor memory, generally this error comes
> when there is not enough memory in individual executors.
> Job is getting completed may be because when tasks are re-scheduled it
> would be going through.
>
> Thanks.
>
> On Mon, 6 Jan 2020 at 5:47 AM, Rishi Shah <rishishah.star@gmail.com>
> wrote:
>
>> Hello All,
>>
>> One of my jobs, keep getting into this situation where 100s of tasks keep
>> failing with below error but job eventually completes.
>>
>> org.apache.spark.memory.SparkOutOfMemoryError: Unable to acquire 16384
>> bytes of memory
>>
>> Could someone advice?
>>
>> --
>> Regards,
>>
>> Rishi Shah
>>
>

-- 
Regards,

Rishi Shah

Mime
View raw message