hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anu Engineer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15007) Stabilize and document Configuration <tag> element
Date Thu, 01 Feb 2018 19:38:01 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349144#comment-16349144

Anu Engineer commented on HADOOP-15007:

I think we are conflating different issues here into this JIRA. So first let us separate the
issues discussed until now and then discuss potential solutions.

# Excessive Logging for Unknown tags: We agree that we should reduce the volume of logging,
and we have two proposals here. First one, log only once for each unknown tag. Second, Log
only in the debug mode. Marton argues that logging in debug mode is useless since no one would
see this warning. 
{{Proposed Solution: Log only once}}.
# Why do we have Enums instead of Strings: The reason is that we use these tags as a set of
known tags -- and use them in the code to display various information to the end user. So
it is a set of well-known strings. That is the use case that enums are designed to solve,
and it is easy for us to detect a tag which we don't know off. Hence the excessive logging
issue. I think if we make it into strings, we will lose the ability even to warn, let us look
for three times or more for a specific string assumes people will not copy/paste wrong tags.
I submit that it is going to happen. So Enums are the best possible way to detect this. With
that, I agree that let us log only once for a tag that we don't know off.
{{Proposed Solution: Convert this to strings or Keep Enums, but fix the original logging issue.}}
# Hadoop Common should not have tags for HDFS and YARN: Now this is a new point and something
that I buy.  
{{Proposed Solution: Load these Enums from the other jars -- and have a config value that
sets the name of the jars or some sort of a function that those JARs can call to register
these tags and it gets used. Let us do that as an improvement to in a different JIRA.}}

if we are doing action item 3, then it is not relevant if we keep this as enums or not. We
still need the logging only once approach to make sure that tags that we don't know of are
warned only once. Does that work for everyone? if so Ajay can start working on a patch.

> Stabilize and document Configuration <tag> element
> --------------------------------------------------
>                 Key: HADOOP-15007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15007
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: conf
>    Affects Versions: 3.1.0
>            Reporter: Steve Loughran
>            Assignee: Ajay Kumar
>            Priority: Blocker
> HDFS-12350 (moved to HADOOP-15005). Adds the ability to tag properties with a <tag>
> We need to make sure that this feature is backwards compatible & usable in production.
That's docs, testing, marshalling etc.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message