drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Igor Guzenko <ihor.huzenko....@gmail.com>
Subject Re: Issue while setting up hive storage plugin in kerberized cluster
Date Fri, 29 Nov 2019 13:00:13 GMT
Hi Rameshwar,

I found difference in your plugin configuration, comparing to
https://mapr.com/docs/51/SecurityGuide/Modify-Hive-Storage-Plugin-In-Drill.html
 .
"hive.server2.enable.doAs": "false" .
Also hive.metastore.kerberos.principal is absent in your configs. In my
opinion it's minor configuration
issue and to resolve it you just need to check every step patiently. Also
maybe some MapR docs could be helpful for you
https://mapr.com/docs/51/SecurityGuide/Configure-Kerberos-Authentication.html
 .

Thanks,
Igor

On Fri, Nov 29, 2019 at 6:24 AM Rameshwar Mane <mr.manerm@gmail.com> wrote:

> The hive metastore is up and running, the metastore is accessible from the
> machine where drill is installed. I have followed all the steps present in
> the documentation.
>
> On Fri, 29 Nov 2019, 05:17 Igor Guzenko, <ihor.huzenko.igs@gmail.com>
> wrote:
>
> > Hello Rameshwar,
> >
> > I think the issue is a little bit more complicated, at least you need to
> be
> > sure that hive metastore is
> > up and running, and is accessible from given machine. Could you please
> > create jira ticket containing
> > configuration info as much as possible ? (hive-site.xml, storage plugin
> > config,  etc.) So then development team could try to reproduce & debug
> > the problem.
> >
> > Thank you in advance,
> > Igor
> >
> > On Thu, Nov 28, 2019 at 7:58 PM Rameshwar Mane <mr.manerm@gmail.com>
> > wrote:
> >
> > > Can anyone provide me any inputs regarding this issue???
> > >
> > > On Thu, 28 Nov 2019, 17:35 Rameshwar Mane, <mr.manerm@gmail.com>
> wrote:
> > >
> > > > Hi all,
> > > >
> > > > I am trying to use drill to query a kerberized cluster. I have setup
> > > drill
> > > > with the provided in documentation for kerberized cluster.
> > > >
> > > > the storage plugin details i am trying to create for hive is :
> > > >
> > > > {
> > > >   "type": "hive",
> > > >   "configProps": {
> > > >     "hive.metastore.uris":
> > > "thrift://xxxx-xxx-xxxx:9083,thrift://xxxx-xxx-xxxx:9083",
> > > >     "hive.metastore.warehouse.dir":
> > "/warehouse/tablespace/managed/hive",
> > > >     "fs.default.name": "hdfs://xxxx-xxx-xxxx:8020",
> > > >     "hive.security.authorization.enabled": "true",
> > > >     "hive.security.authenticator.manager":
> > > "org.apache.hadoop.hive.ql.security.SessionStateUserAuthenticator",
> > > >     "hive.security.authorization.manager":
> > >
> >
> "org.apcahe.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizerFactory",
> > > >     "hive.metastore.sasl.enabled": "true",
> > > >     "hive.server2.enable.doAs": "true"
> > > >     },
> > > >   "enabled": true
> > > > }
> > > >
> > > >
> > > > i am facing the following issue while trying to create hive storage
> > > > plugin:
> > > >
> > > >
> > > >
> > > >
> > > > Please retry: Error while creating / updating storage : Could not
> > > connect to meta store using any of the URIs provided.
> > > >  Most recent failure:
> org.apache.thrift.transport.TTransportException:
> > > >  GSS initiate failed at
> > >
> >
> org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
> > > at
> > org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316)
> > > at
> > > >
> > >
> >
> org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
> > > at
> > >
> >
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
> > > at
> > > >
> > >
> >
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
> > > at java.security.AccessController.doPrivileged(Native Method)
> > > >  at javax.security.auth.Subject.doAs(Subject.java:422) at
> > >
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746)
> > > at
> > > >
> > >
> >
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
> > > at
> > >
> >
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:480)
> > > at
> > > >
> > >
> >
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:247)
> > > at
> > >
> >
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:129)
> > > at
> > > >
> > >
> >
> org.apache.drill.exec.store.hive.client.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:54)
> > > at
> > > >
> > >
> >
> org.apache.drill.exec.store.hive.client.DrillHiveMetaStoreClientFactory.createCloseableClientWithCaching(DrillHiveMetaStoreClientFactory.java:101)
> > > at
> > > >
> > >
> >
> org.apache.drill.exec.store.hive.schema.HiveSchemaFactory.<init>(HiveSchemaFactory.java:77)
> > > at
> > >
> >
> org.apache.drill.exec.store.hive.HiveStoragePlugin.<init>(HiveStoragePlugin.java:77)
> > > at
> > > >  sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> > > at
> > >
> >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> > > at
> > > >
> > >
> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> > > at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at
> > > >
> > >
> >
> org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:466)
> > > at
> > >
> >
> org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(StoragePluginRegistryImpl.java:131)
> > > >  at
> > >
> >
> org.apache.drill.exec.server.rest.PluginConfigWrapper.createOrUpdateInStorage(PluginConfigWrapper.java:56)
> > > at
> > >
> >
> org.apache.drill.exec.server.rest.StorageResources.createOrUpdatePluginJSON(
> > > StorageResources.java:193) at
> > > >
> > >
> >
> org.apache.drill.exec.server.rest.StorageResources.createOrUpdatePlugin(StorageResources.java:210)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> > > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> > > at
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > > at
> > > >  java.lang.reflect.Method.invoke(Method.java:498) at
> > >
> >
> org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
> > > at
> > > >
> > >
> >
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144)
> > > at
> > > >
> > >
> >
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)
> > > at
> > > >
> > >
> >
> org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205)
> > > at
> > > >
> > >
> >
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99)
> > > at
> > > >
> > >
> >
> org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389)
> > > at
> > >
> >
> org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347)
> > > at
> > > >
> > >
> >
> org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102)
> > > at
> > org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326)
> > > at
> > > >  org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) at
> > > org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) at
> > > org.glassfish.jersey.internal.Errors.process(Errors.java:315) at
> > > >  org.glassfish.jersey.internal.Errors.process(Errors.java:297) at
> > > org.glassfish.jersey.internal.Errors.process(Errors.java:267) at
> > > >
> > >
> >
> org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
> > > at
> > >
> >
> org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) at
> > > >
> > >
> >
> org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
> > > at
> > >
> >
> org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473)
> > > at
> > > >
> > >
> org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
> > at
> > >
> >
> org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
> > > at \
> > > >
> > >
> >
> org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
> > > at
> > >
> >
> org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
> > > at
> > > >
> org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
> > > at
> > >
> >
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> > > at
> > > >
> > >
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> > > at
> > >
> >
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
> > > at
> > > >
> > >
> >
> org.apache.drill.exec.server.rest.auth.DrillHttpSecurityHandlerProvider.handle(DrillHttpSecurityHandlerProvider.java:151)
> > > at
> > > >
> > >
> >
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
> > > at
> > >
> >
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
> > > at
> > > >
> > >
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:513)
> > > at
> > >
> >
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> > > at
> > > >
> > >
> >
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
> > > at
> > >
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> > > at
> > > >
> > >
> >
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
> > > at org.eclipse.jetty.server.Server.handle(Server.java:539) at
> > > >  org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333) at
> > >
> >
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
> > > at
> > > >  org.eclipse.jetty.io
> > .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
> > > at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
> at
> > > >  org.eclipse.jetty.io
> > .SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
> > > at
> > > >
> > >
> >
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
> > > at
> > > >
> > >
> >
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
> > > at
> > > >
> > >
> >
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
> > > at
> > > >
> > >
> >
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
> > > at
> > > >
> > >
> >
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
> > > at
> > > >  java.lang.Thread.run(Thread.java:748)
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > am i providing any wrong configurations while creating the storage
> > > plugin?
> > > > I have created drill keytab using the steps provided in the
> > documentation
> > > > and have a mit kerberos autorization service.
> > > >
> > > > let me know any solutions that can be used to solve this.
> > > >
> > > > Thanks and Regards
> > > > *Rameshwar Mane*
> > > > Big Data Engineer
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message