spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From youso b <bouyous...@gmail.com>
Subject unsubscribe
Date Fri, 20 Nov 2020 14:36:29 GMT
On Fri, Nov 20, 2020, 8:25 AM Amit Sharma <resolve123@gmail.com> wrote:

> please help.
>
>
> Thanks
> Amit
>
> On Mon, Nov 9, 2020 at 4:18 PM Amit Sharma <resolve123@gmail.com> wrote:
>
>> Please find below the exact exception
>>
>> Exception in thread "streaming-job-executor-3"
>> java.lang.OutOfMemoryError: Java heap space
>>         at java.util.Arrays.copyOf(Arrays.java:3332)
>>         at
>> java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
>>         at
>> java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448)
>>         at java.lang.StringBuilder.append(StringBuilder.java:136)
>>         at
>> scala.StringContext.standardInterpolator(StringContext.scala:126)
>>         at scala.StringContext.s(StringContext.scala:95)
>>         at sparkStreaming.TRReview.getTRReviews(TRReview.scala:307)
>>         at
>> sparkStreaming.KafkaListener$$anonfun$1$$anonfun$apply$1$$anonfun$3.apply(KafkaListener.scala:154)
>>         at
>> sparkStreaming.KafkaListener$$anonfun$1$$anonfun$apply$1$$anonfun$3.apply(KafkaListener.scala:138)
>>         at scala.util.Success$$anonfun$map$1.apply(Try.scala:237)
>>         at scala.util.Try$.apply(Try.scala:192)
>>         at scala.util.Success.map(Try.scala:237)
>>
>> On Sun, Nov 8, 2020 at 1:35 PM Amit Sharma <resolve123@gmail.com> wrote:
>>
>>> Hi , I am using 16 nodes spark cluster with below config
>>> 1. Executor memory  8 GB
>>> 2. 5 cores per executor
>>> 3. Driver memory 12 GB.
>>>
>>>
>>> We have streaming job. We do not see problem but sometimes we get
>>> exception executor-1 heap memory issue. I am not understanding if data size
>>> is same and this job receive a request and process it but suddenly it’s
>>> start giving out of memory error . It will throw exception for 1 executor
>>> then throw for other executor also and it stop processing the request.
>>>
>>> Thanks
>>> Amit
>>>
>>

Mime
View raw message