spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <sro...@gmail.com>
Subject Re: [DISCUSS] Change default executor log URLs for YARN
Date Sat, 09 Feb 2019 00:18:36 GMT
Is a flag needed? You know me, I think flags are often failures of
design, or disagreement punted to the user. I can understand retaining
old behavior under a flag where the behavior change could be
problematic for some users or facilitate migration, but this is just a
change to some UI links no? the underlying links don't change.
On Fri, Feb 8, 2019 at 5:41 PM Ryan Blue <rblue@netflix.com> wrote:
>
> I suggest using the current behavior as the default and add a flag to implement the behavior
you're suggesting: to link to the logs path in YARN instead of directly to stderr and stdout.
>
> On Fri, Feb 8, 2019 at 3:33 PM Jungtaek Lim <kabhwan@gmail.com> wrote:
>>
>> Ryan,
>>
>> actually I'm not clear about your suggestion. For me three possible options here:
>>
>> 1. If we want to let users be able to completely rewrite log urls, that's SPARK-26792.
For SHS we already addressed it.
>> 2. We could let users turning on/off flag option to just get one url or default two
stdout/stderr urls.
>> 3. We could let users enumerate file names they want to link, and create log links
for each file.
>>
>> Which one do you suggest?
>

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org


Mime
View raw message