hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Miller (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11223) Offer a read-only conf alternative to new Configuration()
Date Fri, 15 Feb 2019 22:08:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16769789#comment-16769789

Michael Miller commented on HADOOP-11223:

{quote}the "Unmodifiable" one is not immutable (see earlier discussion about addDefaultResource).
Correct. I understand a true immutable object is not possible because of the static methods.
This is why we would settle for an "Unmodifiable" object. This would at least prevent changes
on the object itself, for example, in our Testing framework:
public void test(ServerContext context) {
   Configuration conf  = context.getHadoopConf();
   conf.set("prop", "value");
Here we would prefer an error be thrown since the framework expects configuration to be set
a certain way. It is a lot easier to make this mistake then to think "I want to change a property
so I am going to create another configuration, call this static method and reload it ".

> Offer a read-only conf alternative to new Configuration()
> ---------------------------------------------------------
>                 Key: HADOOP-11223
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11223
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: conf
>            Reporter: Gopal V
>            Assignee: Michael Miller
>            Priority: Major
>              Labels: Performance
>         Attachments: HADOOP-11223.001.patch, HADOOP-11223.002.patch, HADOOP-11223.003.patch
> new Configuration() is called from several static blocks across Hadoop.
> This is incredibly inefficient, since each one of those involves primarily XML parsing
at a point where the JIT won't be triggered & interpreter mode is essentially forced on
the JVM.
> The alternate solution would be to offer a {{Configuration::getDefault()}} alternative
which disallows any modifications.
> At the very least, such a method would need to be called from 
> # org.apache.hadoop.io.nativeio.NativeIO::<clinit>()
> # org.apache.hadoop.security.SecurityUtil::<clinit>()
> # org.apache.hadoop.yarn.factory.providers.RecordFactoryProvider::<clinit>

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message