spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ruijing Li <liruijin...@gmail.com>
Subject Re: Can you view thread dumps on spark UI if job finished
Date Wed, 08 Apr 2020 23:50:03 GMT
Thanks Zahid, Yes I am using history server to see previous UIs.

 However, my question still remains on viewing old thread dumps, as I
cannot see them on the old completed spark UIs, only when spark context is
running.

On Wed, Apr 8, 2020 at 4:01 PM Zahid Rahman <zahidr1000@gmail.com> wrote:

> Spark UI is only available while SparkContext is running.
>
> However  You can get to the Spark UI after your application  completes or
> crashes.
>
> To do this Spark includes a tool called the Spark History Server that
> allows you to reconstruct the Spark UI.
>
> You can find up to date information on how to use this tool in the spark
> documentation https://spark.apache.org/docs/latest/monitoring.html
>
>
>
>
>
> On Wed, 8 Apr 2020, 23:47 Ruijing Li, <liruijing09@gmail.com> wrote:
>
>> Hi all,
>>
>> As stated in title, currently when I view the spark UI of a completed
>> spark job, I see there are thread dump links in the executor tab, but
>> clicking on them does nothing. Is it possible to see the thread dumps
>> somehow even if the job finishes? On spark 2.4.5.
>>
>> Thanks.
>> --
>> Cheers,
>> Ruijing Li
>>
> --
Cheers,
Ruijing Li

Mime
View raw message