hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11223) Offer a read-only conf alternative to new Configuration()
Date Mon, 18 Feb 2019 10:57:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16770994#comment-16770994

Steve Loughran commented on HADOOP-11223:

I could probably also modify the config if it was referring to any JCEKS file for passwords,
and I just modified that. What you've got is probably useful in a few places: passing on a
config & you want to prevent accidental changes.

Now, considering patch.

* checkstyle is failing. please fix after making the other changes suggested below
* Javadocs need to clarify this discussion about unmodifieable vers immutable & list ways
where it can be changed underneath.
* checkstyle is unhappy
* Tag new class as @Public @Unstable. We may remove/change it if there are issues. 
* check your ordering of importss, in the hadoop code its






* use {{LambdaTestUtils.intercept()}} for those tests which expect exceptions to be thrown
-we're adopting it in all new code.
* I had a look at {{testAllMethods}} to see if we could do a parameterized junit test suite
from it. I do think it's possible —the reflection code would go into a static {} clause
which would create the parameter table of String method, List<Class> args, which would
then be used to build the test method


> Offer a read-only conf alternative to new Configuration()
> ---------------------------------------------------------
>                 Key: HADOOP-11223
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11223
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: conf
>            Reporter: Gopal V
>            Assignee: Michael Miller
>            Priority: Major
>              Labels: Performance
>         Attachments: HADOOP-11223.001.patch, HADOOP-11223.002.patch, HADOOP-11223.003.patch
> new Configuration() is called from several static blocks across Hadoop.
> This is incredibly inefficient, since each one of those involves primarily XML parsing
at a point where the JIT won't be triggered & interpreter mode is essentially forced on
the JVM.
> The alternate solution would be to offer a {{Configuration::getDefault()}} alternative
which disallows any modifications.
> At the very least, such a method would need to be called from 
> # org.apache.hadoop.io.nativeio.NativeIO::<clinit>()
> # org.apache.hadoop.security.SecurityUtil::<clinit>()
> # org.apache.hadoop.yarn.factory.providers.RecordFactoryProvider::<clinit>

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message