spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacek Laskowski <ja...@japila.pl>
Subject Re: Observing DAGScheduler Log Messages
Date Sun, 07 Apr 2019 19:21:00 GMT
Hi,

Add the following line to conf/log4j.properties and you should have all the
logs:

log4j.logger.org.apache.spark.scheduler.DAGScheduler=ALL

Pozdrawiam,
Jacek Laskowski
----
https://about.me/JacekLaskowski
Mastering Spark SQL https://bit.ly/mastering-spark-sql
Spark Structured Streaming https://bit.ly/spark-structured-streaming
Mastering Kafka Streams https://bit.ly/mastering-kafka-streams
Follow me at https://twitter.com/jaceklaskowski


On Sun, Apr 7, 2019 at 6:05 PM M Bilal <mbilalce.dev@gmail.com> wrote:

> Hi,
>
> I want to observe the log messages from DAGScheduler in Apache Spark.
> Which log files do I need to check.
> I have tried observing the driver logs and worker stderr logs but I can't
> find any messages that are from that class.
>
>  I am using Spark 3.0.0 snapshot in standalone mode.
>
> Thanks.
>
> Regards,
> Bilal
>

Mime
View raw message