hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11771) Configuration::getClassByNameOrNull synchronizes on a static object
Date Sat, 28 Mar 2015 06:38:53 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14385165#comment-14385165

Arun Suresh commented on HADOOP-11771:

[~gopalv], looking at the code, It looks like there is a reason for CACHE_CLASSES to be a
WeakHashMap (to be able to drop the cached classes when the classloader goes away) and there
is a reason why we need it synchronized. 

Id say you could fix this using a {{ConcurrentWeakHashMap}} which we would have to probably
write from scratch (might not be too hard, but I dont expect it to be trivial). Also you have
to note that the actual concurrency level (atleast in the case of {{ConcurrentHashMap}}) has
to be specified in the constructor when it is created, else it will be a default value for
the entirety of its lifetime.

Alternatively, is it possible to reuse a value obtained from an earlier call to this method
by the client code/framework/utility (which based on your stack trace is the hive planner)?

> Configuration::getClassByNameOrNull synchronizes on a static object
> -------------------------------------------------------------------
>                 Key: HADOOP-11771
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11771
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: conf, io, ipc
>            Reporter: Gopal V
>         Attachments: configuration-cache-bt.png, configuration-sync-cache.png
> {code}
>   private static final Map<ClassLoader, Map<String, WeakReference<Class<?>>>>
>     CACHE_CLASSES = new WeakHashMap<ClassLoader, Map<String, WeakReference<Class<?>>>>();
> ...
>  synchronized (CACHE_CLASSES) {
>       map = CACHE_CLASSES.get(classLoader);
>       if (map == null) {
>         map = Collections.synchronizedMap(
>           new WeakHashMap<String, WeakReference<Class<?>>>());
>         CACHE_CLASSES.put(classLoader, map);
>       }
>     }
> {code}
> !configuration-sync-cache.png!
> !configuration-cache-bt.png!

This message was sent by Atlassian JIRA

View raw message