hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Miller (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11223) Offer a read-only conf alternative to new Configuration()
Date Fri, 15 Feb 2019 21:11:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16769739#comment-16769739

Michael Miller commented on HADOOP-11223:

Michael Miller Sorry to join the party so late, just want to understand how Accumulo uses
this feature, as this is still not read-only config, But surely benefits you someway. I would
like to understand the use case a little better. Thanks

Accumulo currently does not use this feature but it is something that would be beneficial.
 We previously had a static class that would save the Configuration object the first time
it was created and then return that object whenever it was needed.  There were a lot of changes
internally to eliminate static state like this and only construct some objects once at startup
of the server.  Mainly for performance and concurrency reasons we don't want the Hadoop Configuration
to change or be re-read.

> Offer a read-only conf alternative to new Configuration()
> ---------------------------------------------------------
>                 Key: HADOOP-11223
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11223
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: conf
>            Reporter: Gopal V
>            Assignee: Michael Miller
>            Priority: Major
>              Labels: Performance
>         Attachments: HADOOP-11223.001.patch, HADOOP-11223.002.patch, HADOOP-11223.003.patch
> new Configuration() is called from several static blocks across Hadoop.
> This is incredibly inefficient, since each one of those involves primarily XML parsing
at a point where the JIT won't be triggered & interpreter mode is essentially forced on
the JVM.
> The alternate solution would be to offer a {{Configuration::getDefault()}} alternative
which disallows any modifications.
> At the very least, such a method would need to be called from 
> # org.apache.hadoop.io.nativeio.NativeIO::<clinit>()
> # org.apache.hadoop.security.SecurityUtil::<clinit>()
> # org.apache.hadoop.yarn.factory.providers.RecordFactoryProvider::<clinit>

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message