drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Igor Guzenko <ihor.huzenko....@gmail.com>
Subject Re: Issue while setting up hive storage plugin in kerberized cluster
Date Thu, 28 Nov 2019 23:47:33 GMT
Hello Rameshwar,

I think the issue is a little bit more complicated, at least you need to be
sure that hive metastore is
up and running, and is accessible from given machine. Could you please
create jira ticket containing
configuration info as much as possible ? (hive-site.xml, storage plugin
config,  etc.) So then development team could try to reproduce & debug
the problem.

Thank you in advance,
Igor

On Thu, Nov 28, 2019 at 7:58 PM Rameshwar Mane <mr.manerm@gmail.com> wrote:

> Can anyone provide me any inputs regarding this issue???
>
> On Thu, 28 Nov 2019, 17:35 Rameshwar Mane, <mr.manerm@gmail.com> wrote:
>
> > Hi all,
> >
> > I am trying to use drill to query a kerberized cluster. I have setup
> drill
> > with the provided in documentation for kerberized cluster.
> >
> > the storage plugin details i am trying to create for hive is :
> >
> > {
> >   "type": "hive",
> >   "configProps": {
> >     "hive.metastore.uris":
> "thrift://xxxx-xxx-xxxx:9083,thrift://xxxx-xxx-xxxx:9083",
> >     "hive.metastore.warehouse.dir": "/warehouse/tablespace/managed/hive",
> >     "fs.default.name": "hdfs://xxxx-xxx-xxxx:8020",
> >     "hive.security.authorization.enabled": "true",
> >     "hive.security.authenticator.manager":
> "org.apache.hadoop.hive.ql.security.SessionStateUserAuthenticator",
> >     "hive.security.authorization.manager":
> "org.apcahe.hadoop.hive.ql.security.authorization.plugin.sqlstd.SQLStdHiveAuthorizerFactory",
> >     "hive.metastore.sasl.enabled": "true",
> >     "hive.server2.enable.doAs": "true"
> >     },
> >   "enabled": true
> > }
> >
> >
> > i am facing the following issue while trying to create hive storage
> > plugin:
> >
> >
> >
> >
> > Please retry: Error while creating / updating storage : Could not
> connect to meta store using any of the URIs provided.
> >  Most recent failure: org.apache.thrift.transport.TTransportException:
> >  GSS initiate failed at
> org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
> at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316)
> at
> >
> org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
> at
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
> at
> >
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
> at java.security.AccessController.doPrivileged(Native Method)
> >  at javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1746)
> at
> >
> org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:480)
> at
> >
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:247)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:129)
> at
> >
> org.apache.drill.exec.store.hive.client.DrillHiveMetaStoreClient.<init>(DrillHiveMetaStoreClient.java:54)
> at
> >
> org.apache.drill.exec.store.hive.client.DrillHiveMetaStoreClientFactory.createCloseableClientWithCaching(DrillHiveMetaStoreClientFactory.java:101)
> at
> >
> org.apache.drill.exec.store.hive.schema.HiveSchemaFactory.<init>(HiveSchemaFactory.java:77)
> at
> org.apache.drill.exec.store.hive.HiveStoragePlugin.<init>(HiveStoragePlugin.java:77)
> at
> >  sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at
> >
> org.apache.drill.exec.store.StoragePluginRegistryImpl.create(StoragePluginRegistryImpl.java:466)
> at
> org.apache.drill.exec.store.StoragePluginRegistryImpl.createOrUpdate(StoragePluginRegistryImpl.java:131)
> >  at
> org.apache.drill.exec.server.rest.PluginConfigWrapper.createOrUpdateInStorage(PluginConfigWrapper.java:56)
> at
> org.apache.drill.exec.server.rest.StorageResources.createOrUpdatePluginJSON(
> StorageResources.java:193) at
> >
> org.apache.drill.exec.server.rest.StorageResources.createOrUpdatePlugin(StorageResources.java:210)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at
> >  java.lang.reflect.Method.invoke(Method.java:498) at
> org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
> at
> >
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144)
> at
> >
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)
> at
> >
> org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205)
> at
> >
> org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99)
> at
> >
> org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389)
> at
> org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347)
> at
> >
> org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102)
> at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326)
> at
> >  org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) at
> org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) at
> org.glassfish.jersey.internal.Errors.process(Errors.java:315) at
> >  org.glassfish.jersey.internal.Errors.process(Errors.java:297) at
> org.glassfish.jersey.internal.Errors.process(Errors.java:267) at
> >
> org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
> at
> org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) at
> >
> org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
> at
> org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473)
> at
> >
> org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) at
> org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
> at \
> >
> org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
> at
> org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
> at
> >  org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:848)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> at
> >
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
> at
> >
> org.apache.drill.exec.server.rest.auth.DrillHttpSecurityHandlerProvider.handle(DrillHttpSecurityHandlerProvider.java:151)
> at
> >
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
> at
> >
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:513)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at
> >
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> >
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
> at org.eclipse.jetty.server.Server.handle(Server.java:539) at
> >  org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333) at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
> at
> >  org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108) at
> >  org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
> at
> >
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
> at
> >
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
> at
> >
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
> at
> >
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
> at
> >
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
> at
> >  java.lang.Thread.run(Thread.java:748)
> >
> >
> >
> >
> >
> > am i providing any wrong configurations while creating the storage
> plugin?
> > I have created drill keytab using the steps provided in the documentation
> > and have a mit kerberos autorization service.
> >
> > let me know any solutions that can be used to solve this.
> >
> > Thanks and Regards
> > *Rameshwar Mane*
> > Big Data Engineer
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message