spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Artur Sukhenko <artur.sukhe...@gmail.com>
Subject Re: event log directory(spark-history) filled by large .inprogress files for spark streaming applications
Date Wed, 17 Jul 2019 15:03:19 GMT
Hi.
There is a workaround for that.
You can disable event logs for Spark Streaming applications.


On Tue, Jul 16, 2019 at 1:08 PM raman gugnani <ramangugnani.007@gmail.com>
wrote:

> HI ,
>
> I have long running spark streaming jobs.
> Event log directories are getting filled with .inprogress files.
> Is there fix or work around for spark streaming.
>
> There is also one jira raised for the same by one reporter.
>
> https://issues.apache.org/jira/browse/SPARK-22783
>
> --
> Raman Gugnani
>
> 8588892293
> Principal Engineer
> *ixigo.com <http://ixigo.com>*
>

Mime
View raw message