spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From M Bilal <mbilalce....@gmail.com>
Subject Re: Observing DAGScheduler Log Messages
Date Mon, 08 Apr 2019 05:26:31 GMT
Thanks Jacek!

On Sun, Apr 7, 2019 at 8:21 PM Jacek Laskowski <jacek@japila.pl> wrote:

> Hi,
>
> Add the following line to conf/log4j.properties and you should have all
> the logs:
>
> log4j.logger.org.apache.spark.scheduler.DAGScheduler=ALL
>
> Pozdrawiam,
> Jacek Laskowski
> ----
> https://about.me/JacekLaskowski
> Mastering Spark SQL https://bit.ly/mastering-spark-sql
> Spark Structured Streaming https://bit.ly/spark-structured-streaming
> Mastering Kafka Streams https://bit.ly/mastering-kafka-streams
> Follow me at https://twitter.com/jaceklaskowski
>
>
> On Sun, Apr 7, 2019 at 6:05 PM M Bilal <mbilalce.dev@gmail.com> wrote:
>
>> Hi,
>>
>> I want to observe the log messages from DAGScheduler in Apache Spark.
>> Which log files do I need to check.
>> I have tried observing the driver logs and worker stderr logs but I can't
>> find any messages that are from that class.
>>
>>  I am using Spark 3.0.0 snapshot in standalone mode.
>>
>> Thanks.
>>
>> Regards,
>> Bilal
>>
>

Mime
View raw message