spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Srinath C <srinat...@gmail.com>
Subject Re: How to get logging right for Spark applications in the YARN ecosystem
Date Fri, 02 Aug 2019 04:46:39 GMT
Hi Raman,

Probably use the rolling file appender in log4j to compress the rotated log
file?

Regards.


On Fri, Aug 2, 2019 at 12:47 AM raman gugnani <ramangugnani.007@gmail.com>
wrote:

> HI ,
>
> I am looking for right solution for logging the logs produced by the
> executors. Most of the places I have seen logging done by log4j properties,
> but no where people I have seen any solution where logs are being
> compressed.
>
> Is there anyway I can compress the logs, So that further those logs can be
> shipped to S3.
>
> --
> Raman Gugnani
>

Mime
View raw message