hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Oleksandr Shevchenko (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15865) ConcurrentModificationException in Configuration.overlay() method
Date Fri, 19 Oct 2018 11:04:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16656643#comment-16656643
] 

Oleksandr Shevchenko commented on HADOOP-15865:
-----------------------------------------------

Thanks [~lqjacklee] for your comment.
Yes, HADOOP-15418 has the same root cause of the problem. Using not thread-safe iterators.

The different is HADOOP-15418 fix the problem related to using iterator of Configuration object
instead of the iterator of Properties object. When we iterate across Properties ConcurrentModificationException
can't be thrown since we work with the local variable.

In the case described in this ticket, we use iterator of Properties object which is the parameter
of overlay() method. As the result, we can get ConcurrentModificationException.

> ConcurrentModificationException in Configuration.overlay() method
> -----------------------------------------------------------------
>
>                 Key: HADOOP-15865
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15865
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Oleksandr Shevchenko
>            Assignee: Oleksandr Shevchenko
>            Priority: Major
>         Attachments: HADOOP-15865.001.patch
>
>
> Configuration.overlay() is not thread-safe and can be the cause of ConcurrentModificationException
since we use iteration over Properties object. 
> {code}
> private void overlay(Properties to, Properties from) {
>  for (Entry<Object, Object> entry: from.entrySet()) {
>  to.put(entry.getKey(), entry.getValue());
>  }
>  }
> {code}
> Properties class is thread-safe but iterator is not. We should manually synchronize on
the returned set of entries which we use for iteration.
> We faced with ResourceManger fails during recovery caused by ConcurrentModificationException:
> {noformat}
> 2018-10-12 08:00:56,968 INFO org.apache.hadoop.service.AbstractService: Service ResourceManager
failed in state STARTED; cause: java.util.ConcurrentModificationException
> java.util.ConcurrentModificationException
>  at java.util.Hashtable$Enumerator.next(Hashtable.java:1383)
>  at org.apache.hadoop.conf.Configuration.overlay(Configuration.java:2801)
>  at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2696)
>  at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2632)
>  at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2528)
>  at org.apache.hadoop.conf.Configuration.get(Configuration.java:1062)
>  at org.apache.hadoop.conf.Configuration.getStringCollection(Configuration.java:1914)
>  at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:53)
>  at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:2043)
>  at org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:2023)
>  at org.apache.hadoop.yarn.webapp.util.WebAppUtils.getPassword(WebAppUtils.java:452)
>  at org.apache.hadoop.yarn.webapp.util.WebAppUtils.loadSslConfiguration(WebAppUtils.java:428)
>  at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:293)
>  at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startWepApp(ResourceManager.java:1017)
>  at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1117)
>  at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
>  at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1251)
> 2018-10-12 08:00:56,968 INFO org.apache.hadoop.yarn.server.resourcemanager.security.RMDelegationTokenSecretManager:
removing RMDelegation token with sequence number: 3489914
> 2018-10-12 08:00:56,968 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore:
Removing RMDelegationToken and SequenceNumber
> 2018-10-12 08:00:56,968 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.FileSystemRMStateStore:
Removing RMDelegationToken_3489914
> 2018-10-12 08:00:56,969 INFO org.apache.hadoop.ipc.Server: Stopping server on 8032
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message