sentry-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hao Hao (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SENTRY-957) Exceptions in MetastoreCacheInitializer should probably not prevent HMS from starting up
Date Fri, 18 Dec 2015 01:28:46 GMT

     [ https://issues.apache.org/jira/browse/SENTRY-957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Hao Hao updated SENTRY-957:
---------------------------
    Description: 
Right now if there is an error condition in evaluating paths in MetastoreCacheInitializer
in one of the tasks, we throw a Runtime Exception (in Sentry-888). Changed the behavior to
be:

* Retry the failed tasks x times. Each retry has a y millis wait duration. x and y are based
upon the user config (SENTRY_HDFS_SYNC_METASTORE_CACHE_RETRY_MAX_NUM and SENTRY_HDFS_SYNC_METASTORE_CACHE_RETRY_WAIT_DURAION_IN_MILLIS).
* After retry failure, throw exception or sync incomplete paths update based on user config
(SENTRY_HDFS_SYNC_METASTORE_CACHE_FAIL_ON_PARTIAL_UPDATE). The default values is fail on partial
update, which will throw runtime exception.

{noformat}
for (Future<CallResult> result : results) {
      CallResult callResult = result.get();

      // Fail the HMS startup if tasks are not all successful and
      // fail on partial updates flag is set in the config.
      if (callResult.getSuccessStatus() == false && failOnRetry) {
        throw new RuntimeException(callResult.getFailure());
      }
    }
{noformat}

  was:
Right now if there is an error condition in evaluating paths in MetastoreCacheInitializer
in one of the tasks, we throw a Runtime Exception (in Sentry-888). Changed the behavior to
be It looks like it would be best to log an error and continue; as a plugin should not cause
the service to come down? On the other hand, doing so might result in incomplete sync which
might make it hard to get to the root cause. 
{noformat}
for (Future<CallResult> result : results) {
      CallResult callResult = result.get();
      if (callResult.failure != null) {
        throw new RuntimeException(callResult.failure);
      }
    }
{noformat}


> Exceptions in MetastoreCacheInitializer should probably not prevent HMS from starting
up
> ----------------------------------------------------------------------------------------
>
>                 Key: SENTRY-957
>                 URL: https://issues.apache.org/jira/browse/SENTRY-957
>             Project: Sentry
>          Issue Type: Bug
>          Components: Hdfs Plugin
>    Affects Versions: 1.7.0
>            Reporter: Sravya Tirukkovalur
>            Assignee: Hao Hao
>             Fix For: 1.7.0
>
>         Attachments: SENTRY-957.001.patch, SENTRY-957.002.patch, SENTRY-957.003.patch
>
>
> Right now if there is an error condition in evaluating paths in MetastoreCacheInitializer
in one of the tasks, we throw a Runtime Exception (in Sentry-888). Changed the behavior to
be:
> * Retry the failed tasks x times. Each retry has a y millis wait duration. x and y are
based upon the user config (SENTRY_HDFS_SYNC_METASTORE_CACHE_RETRY_MAX_NUM and SENTRY_HDFS_SYNC_METASTORE_CACHE_RETRY_WAIT_DURAION_IN_MILLIS).
> * After retry failure, throw exception or sync incomplete paths update based on user
config (SENTRY_HDFS_SYNC_METASTORE_CACHE_FAIL_ON_PARTIAL_UPDATE). The default values is fail
on partial update, which will throw runtime exception.
> {noformat}
> for (Future<CallResult> result : results) {
>       CallResult callResult = result.get();
>       // Fail the HMS startup if tasks are not all successful and
>       // fail on partial updates flag is set in the config.
>       if (callResult.getSuccessStatus() == false && failOnRetry) {
>         throw new RuntimeException(callResult.getFailure());
>       }
>     }
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message