spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Srinivas V <srini....@gmail.com>
Subject Re: Kafka spark structure streaming out of memory issue
Date Fri, 14 Aug 2020 03:16:34 GMT
It depends on how much memory is available and how much data you are
processing. Please provide data size and cluster details to help.

On Fri, Aug 14, 2020 at 12:54 AM km.santanu <Km4.santanu@gmail.com> wrote:

> Hi
> I am using Kafka stateless structure streaming.i have enabled watermark as
> 1
> hour.after long running about 2 hour my job is terminating
> automatically.check point has been enabled.
> I am doing average on input data.
> Can you please suggest how to avoid out of memory error
>
>
>
> --
> Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>

Mime
View raw message