hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Levas (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-16340) ABFS driver continues to retry on IOException responses from REST operations
Date Fri, 31 May 2019 17:46:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-16340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Robert Levas updated HADOOP-16340:
----------------------------------
    Description: 
ABFS driver continues to retry (until retry count is exhausted) upon IOException responses
from REST operations.  

In the exception hander for IOExceptions at [https://github.com/apache/hadoop/blob/65f60e56b082faf92e1cd3daee2569d8fc669c67/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java#L174-L197],
there is no way exit out of the retry loop by re-throwing an exception unless one of the
following conditions have been met:
 * The retry limit was hit
 * An HttpException was encountered

>From an {{org.apache.hadoop.fs.azurebfs.extensions.CustomTokenProviderAdaptee}} or {{org.apache.hadoop.fs.azurebfs.extensions.CustomDelegationTokenManager}}
implementation, there is no way to create an {{org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.HttpException}}
since the constructor is package private. 

Either the exception handler needs to generic handle exceptions like {{java.nio.file.AccessDeniedException}}
and {{java.io.FileNotFoundException}}, or the access to {{org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.HttpException}}
needs to be set to that custom implementations can use it. 

 

  was:
ABFS driver continues to retry (until retry count is exhausted) upon access denied responses
from REST operations.  

In the exception hander for IOExceptions at [https://github.com/apache/hadoop/blob/65f60e56b082faf92e1cd3daee2569d8fc669c67/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java#L174-L197],
there is no way exit out of the retry loop by re-throwing an exception unless one of the
following conditions have been met:
 * The retry limit was hit
 * An HttpException was encountered

>From an {{org.apache.hadoop.fs.azurebfs.extensions.CustomTokenProviderAdaptee}} or {{org.apache.hadoop.fs.azurebfs.extensions.CustomDelegationTokenManager}}
implementation, there is no way to create an {{org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.HttpException}}
since the constructor is package private. 

Either the exception handler needs to generic handle exceptions like {{java.nio.file.AccessDeniedException}}
and {{java.io.FileNotFoundException}}, or the access to {{org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.HttpException}}
needs to be set to that custom implementations can use it. 

 


> ABFS driver continues to retry on IOException responses from REST operations
> ----------------------------------------------------------------------------
>
>                 Key: HADOOP-16340
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16340
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/azure
>    Affects Versions: 3.3.0
>            Reporter: Robert Levas
>            Priority: Major
>
> ABFS driver continues to retry (until retry count is exhausted) upon IOException responses
from REST operations.  
> In the exception hander for IOExceptions at [https://github.com/apache/hadoop/blob/65f60e56b082faf92e1cd3daee2569d8fc669c67/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsRestOperation.java#L174-L197],
there is no way exit out of the retry loop by re-throwing an exception unless one of the
following conditions have been met:
>  * The retry limit was hit
>  * An HttpException was encountered
> From an {{org.apache.hadoop.fs.azurebfs.extensions.CustomTokenProviderAdaptee}} or {{org.apache.hadoop.fs.azurebfs.extensions.CustomDelegationTokenManager}}
implementation, there is no way to create an {{org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.HttpException}}
since the constructor is package private. 
> Either the exception handler needs to generic handle exceptions like {{java.nio.file.AccessDeniedException}}
and {{java.io.FileNotFoundException}}, or the access to {{org.apache.hadoop.fs.azurebfs.oauth2.AzureADAuthenticator.HttpException}}
needs to be set to that custom implementations can use it. 
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message