hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Piotr Wikieł (JIRA) <>
Subject [jira] [Commented] (HIVE-13075) Metastore shuts down when no delegation token is found in ZooKeeper
Date Mon, 22 Feb 2016 17:35:18 GMT


Piotr Wikieł commented on HIVE-13075:

Thank you, I'll remember that. 

My comment was about need of small improvements in "How to contribute" document. For non-english
speakers, like me, a to-do list from "I've got some code" to "see my code after git pull on
master" would be invaluable ;)

> Metastore shuts down when no delegation token is found in ZooKeeper
> -------------------------------------------------------------------
>                 Key: HIVE-13075
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 1.3.0, 1.2.1
>            Reporter: Piotr Wikieł
>            Priority: Minor
> {{ZooKeeperTokenStore}} looks [as follows|]:
> {code:java}
> @Override
> public DelegationTokenInformation getToken(DelegationTokenIdentifier tokenIdentifier)
>   byte[] tokenBytes = zkGetData(getTokenPath(tokenIdentifier));
>   try {
>     return HiveDelegationTokenSupport.decodeDelegationTokenInformation(tokenBytes);
>   } catch (Exception ex) {
>     throw new TokenStoreException("Failed to decode token", ex);
>   }
> }
> {code}
> which is slightly different from [DBTokenStore implementation|]
that is protected against {{tokenBytes==null}} because nullable {{tokenBytes}} causes NPE
to be thrown in [HiveDelegationTokenSupport#decodeDelegationTokenInformation|]
> Furthermore, NPE thrown here causes [TokenStoreDelegationTokenSecretManager.ExpiredTokenRemover|]
to catch it and exits MetaStore.
> {{null}} from {{[zkGetData()|]}}
is possible during ZooKeeper failure or (and that was our case) when another metastore instance
removes tokens during {{ExpiredTokenRemover}} run. There were two solutions of this problem:
>  * distributed lock in ZooKeeper acquired during one metastore instance's ExpiredTokenRemover
>  * simple null check
> I think null check is sufficient if it is in {{DBTokenStore}}.
> Patch will be attached.
> Sorry for an edit but I think worth mentioning is a fact that possible workaround for
this issue is setting {{hive.cluster.delegation.key.update-interval}}, {{hive.cluster.delegation.token.renew-interval}}
and {{hive.cluster.delegation.token.max-lifetime}} to one year as described [here|].
But in my opinion it is not an engineer-way of doing things ;)

This message was sent by Atlassian JIRA

View raw message