hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [hadoop] elek commented on issue #773: HDDS-1469. Generate default configuration fragments based on annotations
Date Fri, 26 Apr 2019 10:02:52 GMT
elek commented on issue #773: HDDS-1469. Generate default configuration fragments based on
annotations
URL: https://github.com/apache/hadoop/pull/773#issuecomment-487003326
 
 
   > I am ok with that, but some of the old school people might like a single file, and
in the deployment, phase don't we need a single file ? or should we move away since the code
already has the default?
   
   It's a very good question and I don't know the final answer. In fact we use standard hadoop
Configuration features to load all the fragments, so it should be fine. I would prefer to
try out this approach (with independent config fragments), but based on the feedback, experiences,
we can improve/refactor it.
   
   My arguments:
   
    1. First of all, it's easier to implement. We don't need a final merge.
    2. It's way easier to test. To generate the final ozone-default.xml we need a project
which depends on all the others with config fragments. But in the mean time we need merged
ozone-default.xml to test the different components. With fragments it just works based on
classpath.
    3. The biggest argument to use one ozone-default.xml (IMHO) is that it can be used as
a documentation. But I think we can provide better documentation page (with better structures).
But it can be true: we may need to generate a static doc page about all the configuration
settings.
    4. It's very interesting that the source of a key is recorded in the Configuration class.
With using fragments we will have a source information out of the box:
   
   
   ```XML
   <property>
      <name>hdds.scm.replication.event.timeout</name>
      <value>10m</value>
      <final>false</final>
      <source>jar:file:/opt/hadoop/share/ozone/lib/hadoop-hdds-server-scm-0.5.0-SNAPSHOT.jar!/ozone-default-generated.xml</source>
   </property>
   ```
   
   It also means that we don't need to use SCM, HDDS, OZONE tags any more as they can be added
based on the source. And with this approach we can print out the configuration based on the
components (eg. SCM configs, common configs, etc.). Would be great to add an other information,
too: which class defined the specific configuration key.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message