sentry-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject [08/52] [abbrv] sentry git commit: CDH-53414: Sentry fails to setup secure connection to HMS if the service unix user is missing active tgt
Date Wed, 14 Jun 2017 00:56:46 GMT
CDH-53414: Sentry fails to setup secure connection to HMS if the service unix user is missing
active tgt

Looking at the code of Sentry and HiveMetaStoreClient, it looks like
this problem has been there for a long time. But it's mostly masked by
the restarts we are doing after the clean deployment.

Some context on how our internal deployment works:
CDEP as part of deployment kicks off the first run of all services via
CM and once it is done, it kinit's service principals under the service
unix users for e.g: hdfs@VPC.CLOUDERA.COM gets activated under hdfs

So during the course of CM first run, sentry and other services don't
have any keytabs active under local unix user and during this time, the
log of sentry is cluttered with the above errors. But after we do a
restart after CDEP completes, the Sentry code picks up the principal
that was activated under "sentry" user and thus works properly.

Ideally the service processes shouldn't depend on the keytabs active on
running unix user but rather use the keytabs supplied by CM in the
process directory.

Looking at the code of HMSFollower we invoke the classes of Hive -
HiveMetaStoreClient which exclusively uses the UserGroupInformation
object. And for UserGroupInformation object to pick up the keytab, one
should explicitly call the methods like loginUserFromKeytabAndReturnUGI
or use getUGIFromSubject. For the fact that HMSFollower already logs
into the keytab, we can leverage the method getUGIFromSubject() to make
the UserGroupInformation aware of keytab authentication.

If UGI object is not made aware of keytab, then the invocation of
UserGroupInformation#getCurrentUser falls back to Unix user.

Adding extra logic to HMSFollower to make UserGroupInformation aware
of the keytab that way it uses it for communication with HMS.

Change-Id: I520ef1183b55e286386d401ce6c1b3946a648d18
Reviewed-by: Alexander Kolbasov <>
Tested-by: Alexander Kolbasov <>


Branch: refs/for/cdh5-1.5.1_ha
Commit: ee488318228b454e0398ab9225ef9bbd7a63624c
Parents: 3f9e25e
Author: Vamsee Yarlagadda <>
Authored: Wed May 3 21:55:50 2017 -0700
Committer: Alexander Kolbasov <>
Committed: Thu May 4 00:06:17 2017 -0700

 .../org/apache/sentry/service/thrift/  | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)
diff --git a/sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/
index 122da84..c3f821c 100644
--- a/sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/
+++ b/sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/service/thrift/
@@ -30,6 +30,7 @@ import org.apache.hadoop.hive.metastore.api.NotificationEventResponse;
 import org.apache.hive.hcatalog.messaging.HCatEventMessage;
 import org.apache.sentry.binding.hive.conf.HiveAuthzConf;
 import org.apache.sentry.core.common.exception.SentryInvalidHMSEventException;
@@ -46,12 +47,10 @@ import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 import org.apache.sentry.binding.metastore.messaging.json.*;
 import java.util.List;
 import java.util.Map;
@@ -118,7 +117,7 @@ public class HMSFollower implements Runnable {
    * Throws @MetaException if there was a problem on creating an HMSClient
   private HiveMetaStoreClient getMetaStoreClient(Configuration conf)
-      throws LoginException, MetaException, PrivilegedActionException {
+      throws IOException, InterruptedException, LoginException, MetaException {
     if(client != null) {
       return client;
@@ -159,9 +158,12 @@ public class HMSFollower implements Runnable {
         // Instantiating SentryKerberosContext in non-server mode handles the ticket renewal.
         kerberosContext = new SentryKerberosContext(principal, keytab, false);
+        UserGroupInformation.setConfiguration(hiveConf);
+        UserGroupInformation clientUGI = UserGroupInformation.getUGIFromSubject(kerberosContext.getSubject());
         // HiveMetaStoreClient handles the connection retry logic to HMS and can be configured
using properties:
         // hive.metastore.connect.retries, hive.metastore.client.connect.retry.delay
-        client = Subject.doAs(kerberosContext.getSubject(), new PrivilegedExceptionAction<HiveMetaStoreClient>()
+        client = clientUGI.doAs(new PrivilegedExceptionAction<HiveMetaStoreClient>()
           public HiveMetaStoreClient run() throws Exception {
             return new HiveMetaStoreClient(hiveConf);
@@ -172,9 +174,6 @@ public class HMSFollower implements Runnable {
         // Kerberos login failed
         LOGGER.error("Failed to setup kerberos context.");
         throw e;
-      } catch (PrivilegedActionException e) {
-        LOGGER.error("Failed to setup secure connection to HMS.");
-        throw e;
       } finally {
         // Shutdown kerberos context if HMS connection failed to setup to avoid thread leaks.
         if ((kerberosContext != null) && (client == null)) {

View raw message