hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adam Antal (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-16580) Disable retry of FailoverOnNetworkExceptionRetry in case of AccessControlException
Date Wed, 18 Sep 2019 14:49:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-16580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16932522#comment-16932522
] 

Adam Antal commented on HADOOP-16580:
-------------------------------------

Thanks for the review [~shuzirra], let me give you a bit more clarification on the patch.

UnreliableInterface is an interface for *purely testing purpose*. It is used for testing RetryPolicy
applied to the UnreliableInterface which mimics an interface that is "unreliable" in a sense
that it might throws exceptions. It is used for checking that the RetryPolicy is triggered
/ works as expected having an unreliable underlying resource (UnreliableImplementation) throwing
Exception in particular circumstances.

The annotation is something that is needed for the imitation - the API calls (like {{mapred
job -list}} - see HADOOP-14982) are usually annotated like that, and in the code there's a
part where:
{code:java}
...
 } else if (e instanceof SocketException
          || (e instanceof IOException && !(e instanceof RemoteException))) {
        if (isIdempotentOrAtMostOnce) {
          return new RetryAction(RetryAction.RetryDecision.FAILOVER_AND_RETRY,
              getFailoverOrRetrySleepTime(retries));
        } else {
          return new RetryAction(RetryAction.RetryDecision.FAIL, 0,
              "the invoked method is not idempotent, and unable to determine "
                  + "whether it was invoked");
        }
      } ...
{code}
where the {{isIdempotentOrAtMostOnce}} variable is determined by the interface's annotation.
Through reflection it is queried whether the method has the {{@AtMostOnce}} or {{@Idempotent}}
annotation and the {{FailoverOnNetworkExceptionRetry}} RetryPolicy decides accordingly to
retry or fail.

Since I wanted to test the RetryPolicy on an Idempotent-annotated call (just like the API
call) which throws an AccessControlException I think the test is logically correct, and the
annotation is proper for this reason. 

As for the name of the method:
The counter starts from 0, and each time the function is called it is incremented by one and
will throw exception each time until it reaches 8 (which is the 9th call) where it won't emit
any new exception. I think this is the exact behaviour that the name suggests. If there's
any other issue or adding some javadoc to make this clearer I can work on that. It was hard
to understand me these concepts first since the lack of javadoc, but hope the above description
helps.

> Disable retry of FailoverOnNetworkExceptionRetry in case of AccessControlException
> ----------------------------------------------------------------------------------
>
>                 Key: HADOOP-16580
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16580
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: common
>    Affects Versions: 3.3.0
>            Reporter: Adam Antal
>            Assignee: Adam Antal
>            Priority: Major
>         Attachments: HADOOP-16580.001.patch, HADOOP-16580.002.patch
>
>
> HADOOP-14982 handled the case where a SaslException is thrown. The issue still persists,
since the exception that is thrown is an *AccessControlException* because user has no kerberos
credentials. 
> My suggestion is that we should add this case as well to {{FailoverOnNetworkExceptionRetry}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message