sentry-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hao Hao (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SENTRY-957) Exceptions in MetastoreCacheInitializer should probably not prevent HMS from starting up
Date Sat, 19 Dec 2015 00:31:46 GMT

     [ https://issues.apache.org/jira/browse/SENTRY-957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Hao Hao updated SENTRY-957:
---------------------------
    Description: 
Right now if there is an error condition in evaluating paths in MetastoreCacheInitializer
in one of the tasks, we throw a Runtime Exception (in Sentry-888). Changed the behavior to
be:

* Retry the failed tasks x times. Each retry has a y millis wait duration. x and y are based
upon the user config (sentry.hdfs.sync.metastore.cache.retry.max.num and sentry.hdfs.sync.metastore.cache.retry.wait.duration.millis).
* After retry failure, throw exception or sync incomplete paths update based on user config
(sentry.hdfs.sync.metastore.cache.fail.on.partial.update). The default values is fail on partial
update, which will throw runtime exception.

{noformat}
for (Future<CallResult> result : results) {
      CallResult callResult = result.get();

      // Fail the HMS startup if tasks are not all successful and
      // fail on partial updates flag is set in the config.
      if (callResult.getSuccessStatus() == false && failOnRetry) {
        throw new RuntimeException(callResult.getFailure());
      }
    }
{noformat}

  was:
Right now if there is an error condition in evaluating paths in MetastoreCacheInitializer
in one of the tasks, we throw a Runtime Exception (in Sentry-888). Changed the behavior to
be:

* Retry the failed tasks x times. Each retry has a y millis wait duration. x and y are based
upon the user config (SENTRY_HDFS_SYNC_METASTORE_CACHE_RETRY_MAX_NUM and SENTRY_HDFS_SYNC_METASTORE_CACHE_RETRY_WAIT_DURAION_IN_MILLIS).
* After retry failure, throw exception or sync incomplete paths update based on user config
(SENTRY_HDFS_SYNC_METASTORE_CACHE_FAIL_ON_PARTIAL_UPDATE). The default values is fail on partial
update, which will throw runtime exception.

{noformat}
for (Future<CallResult> result : results) {
      CallResult callResult = result.get();

      // Fail the HMS startup if tasks are not all successful and
      // fail on partial updates flag is set in the config.
      if (callResult.getSuccessStatus() == false && failOnRetry) {
        throw new RuntimeException(callResult.getFailure());
      }
    }
{noformat}


> Exceptions in MetastoreCacheInitializer should probably not prevent HMS from starting
up
> ----------------------------------------------------------------------------------------
>
>                 Key: SENTRY-957
>                 URL: https://issues.apache.org/jira/browse/SENTRY-957
>             Project: Sentry
>          Issue Type: Bug
>          Components: Hdfs Plugin
>    Affects Versions: 1.7.0
>            Reporter: Sravya Tirukkovalur
>            Assignee: Hao Hao
>             Fix For: 1.7.0
>
>         Attachments: SENTRY-957.001.patch, SENTRY-957.002.patch, SENTRY-957.003.patch
>
>
> Right now if there is an error condition in evaluating paths in MetastoreCacheInitializer
in one of the tasks, we throw a Runtime Exception (in Sentry-888). Changed the behavior to
be:
> * Retry the failed tasks x times. Each retry has a y millis wait duration. x and y are
based upon the user config (sentry.hdfs.sync.metastore.cache.retry.max.num and sentry.hdfs.sync.metastore.cache.retry.wait.duration.millis).
> * After retry failure, throw exception or sync incomplete paths update based on user
config (sentry.hdfs.sync.metastore.cache.fail.on.partial.update). The default values is fail
on partial update, which will throw runtime exception.
> {noformat}
> for (Future<CallResult> result : results) {
>       CallResult callResult = result.get();
>       // Fail the HMS startup if tasks are not all successful and
>       // fail on partial updates flag is set in the config.
>       if (callResult.getSuccessStatus() == false && failOnRetry) {
>         throw new RuntimeException(callResult.getFailure());
>       }
>     }
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message