hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15007) Stabilize and document Configuration <tag> element
Date Wed, 01 Nov 2017 16:34:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16234337#comment-16234337

Steve Loughran commented on HADOOP-15007:

# Is this expected to be a stable change? Or at least: when do we expect it to be stable?
# What is going to happen when an older version of Hadoop encounters an XML file with a new
field in it?
# we need some tests which actually set out to break the mechanism. Invalid tags etc.
# Javadocs and end-user docs to cover what can and cannot be done with tags, how to use. {{Configuration's}}
own javadocs are the defacto documentation of the XML format: they need to be updated with
the change.
# what's going to to happen when existing code which serializes/deserializes configs using
Hadoop writables encounters configs with tags? Can an old hadoop-common lib deserialize, say
core-default.xml with tags added? I don't see any tests for that, and I'm assuming "no" unless
it can be demonstrated.
# Can I add tags to a propery retrieved with getPassword()? 
# If I overrride a -default property in a file, does it inherit the tags from the parent?
# If I add tags to an overidden property, do the tags override or replace existing ones?
# {{Configuration.readTagFromConfig}}. If there's an invalid tag in a -default file, is it
going to fill the log files with info messages on every load of every single configuration
file? If so, it's too noisy. One warning per (file/property) the way we do for deprecated
# {{Configuration.getPropertyTag}} returns a {{PropertyTag}} enum; which is marked as Private/Evolving.
But configuration is Public/Stable. Either we need to mark PropertyTag as Public/{Evolving/Unstable}
or {{getPropertyTag}} is marked as Private
# Could there have been a way to allow {{PropertyTag}} enums to be registered dynamically/via
classname strings, so that they can be kept in the specific modules. We've now tainted hadoop-common
with details about yarn, hdfs, ozone.

Right now, I'm not convinced this is a compatible change, and will need to be convinced that
it is before we can ship a version of Hadoop with the feature. If there are problems, unless
they can be addressed we may need to exclude supporting tags in the core-, hdfs- and yarn-
on compatibility grounds.

> Stabilize and document Configuration <tag> element
> --------------------------------------------------
>                 Key: HADOOP-15007
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15007
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: conf
>    Affects Versions: 3.1.0
>            Reporter: Steve Loughran
>            Priority: Blocker
> HDFS-12350 (moved to HADOOP-15005). Adds the ability to tag properties with a <tag>
> We need to make sure that this feature is backwards compatible & usable in production.
That's docs, testing, marshalling etc.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message