hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [hadoop] elek commented on a change in pull request #773: HDDS-1469. Generate default configuration fragments based on annotations
Date Fri, 26 Apr 2019 09:32:09 GMT
elek commented on a change in pull request #773: HDDS-1469. Generate default configuration
fragments based on annotations
URL: https://github.com/apache/hadoop/pull/773#discussion_r278874474
 
 

 ##########
 File path: hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/conf/OzoneConfiguration.java
 ##########
 @@ -161,14 +285,14 @@ public static void activate() {
     Configuration.addDefaultResource("hdfs-default.xml");
     Configuration.addDefaultResource("hdfs-site.xml");
     Configuration.addDefaultResource("ozone-default.xml");
-    Configuration.addDefaultResource("ozone-site.xml");
 
 Review comment:
   We should. But in fact I didn't remove the possibility to use ozone-site.xml I just _moved_
it.
   
   The problem here is that the hadoop Configuration is not designed to be reused. We can
use multiple `Configuration.addDefaultResource(String)` but there is no `Configuration.addDefaultResource(URL)`
(and we can't create one as all the required methods are private :-( ).
   
   With using configuration fragments we have multiple files with the same name (String) but
with different URL (eg. jar://hadoop-ozone-ozone-manager.jar!/ozone-default-generated.xml).
   
   To support this, but _keep_ the same precedence I modified the configuration loading:
   
   ```
   Configuration.addDefaultResource("hdfs-default.xml");
   Configuration.addDefaultResource("hdfs-site.xml");
   Configuration.addDefaultResource("ozone-default.xml");
   
   conf.addResource("jar://ozone-manager.jar!/ozone-site-generated.xml");
   conf.addResource("jar://storage-container-manager.jar!/ozone-site-generated.xml");
   
   conf.addResource("ozone-site.xml");
   ```
   
   The last three lines are in the constructor of OzoneConfiguration and as I would like to
load `ozone-site.xml ` at the end I moved it to there.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message