spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <>
Subject Re: [DISCUSS] Change default executor log URLs for YARN
Date Sat, 09 Feb 2019 00:18:36 GMT
Is a flag needed? You know me, I think flags are often failures of
design, or disagreement punted to the user. I can understand retaining
old behavior under a flag where the behavior change could be
problematic for some users or facilitate migration, but this is just a
change to some UI links no? the underlying links don't change.
On Fri, Feb 8, 2019 at 5:41 PM Ryan Blue <> wrote:
> I suggest using the current behavior as the default and add a flag to implement the behavior
you're suggesting: to link to the logs path in YARN instead of directly to stderr and stdout.
> On Fri, Feb 8, 2019 at 3:33 PM Jungtaek Lim <> wrote:
>> Ryan,
>> actually I'm not clear about your suggestion. For me three possible options here:
>> 1. If we want to let users be able to completely rewrite log urls, that's SPARK-26792.
For SHS we already addressed it.
>> 2. We could let users turning on/off flag option to just get one url or default two
stdout/stderr urls.
>> 3. We could let users enumerate file names they want to link, and create log links
for each file.
>> Which one do you suggest?

To unsubscribe e-mail:

View raw message