atlas-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Madhan Neethiraj (Jira)" <j...@apache.org>
Subject [jira] [Resolved] (ATLAS-391) Hive hook fails when capturing create/alter table in hive
Date Mon, 18 May 2020 01:46:00 GMT

     [ https://issues.apache.org/jira/browse/ATLAS-391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Madhan Neethiraj resolved ATLAS-391.
------------------------------------
    Resolution: Abandoned

The referenced code doesn't exist in Atlas any more.

> Hive hook fails when capturing create/alter table in hive
> ---------------------------------------------------------
>
>                 Key: ATLAS-391
>                 URL: https://issues.apache.org/jira/browse/ATLAS-391
>             Project: Atlas
>          Issue Type: Bug
>    Affects Versions: 0.5-incubating
>         Environment: HDP-2.3.2.0-2950 hdp
> centOS 6.7
>            Reporter: John Mattucci
>            Priority: Critical
>
> Getting the following error as a result of attempting  to create/alter hive table and
the hive hook fails to create hive_table type in Atlas 
> Please note when I attempt to run the query
> ...:21000/api/atlas/discovery/search/dsl?query= hive_db where name = 'tsz' and clusterName
= 'primary' it fails with the same error however if I modify the query and alter one of param
letters to uppercase it runs fine ...21000/api/atlas/discovery/search/dsl?query= hive_db where
name = 'Tsz' and clusterName = 'primary'
> In the atlas application.log we see the following 
> 2015-12-15 13:29:06,200 ERROR - [346859523@qtp-240063939-34 - 25e162ee-56c1-4cb1-ac3a-65f472489c21:]
~ graph rollback due to exception  (GraphTransactionInterceptor:47)
> org.apache.atlas.discovery.DiscoveryException: Invalid expression : hive_db where name
= 'tsz' and clusterName = 'primary'
>         at org.apache.atlas.discovery.graph.GraphBackedDiscoveryService.evaluate(GraphBackedDiscoveryService.java:140)
>         at org.apache.atlas.discovery.graph.GraphBackedDiscoveryService.searchByDSL(GraphBackedDiscoveryService.java:124)
>         at org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:41)
>         at org.apache.atlas.web.resources.MetadataDiscoveryResource.searchUsingQueryDSL(MetadataDiscoveryResource.java:128)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:606)
>         at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
>         at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
>         at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
>         at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
>         at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
>         at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
>         at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
>         at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
>         at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
>         at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
>         at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
>         at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
>         at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
>         at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
>         at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>         at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:287)
>         at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:277)
>         at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:182)
>         at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
>         at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
>         at org.apache.atlas.web.filters.AuditFilter.doFilter(AuditFilter.java:67)
>         at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
>         at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:119)
>         at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:133)
>         at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:130)
>         at com.google.inject.servlet.GuiceFilter$Context.call(GuiceFilter.java:203)
>         at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:130)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
>         at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>         at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
>         at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
>         at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
>         at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>         at org.mortbay.jetty.Server.handle(Server.java:326)
>         at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
>         at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
>         at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
>         at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
>         at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>         at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
>         at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
> Caused by: java.lang.NullPointerException
>         at org.apache.atlas.repository.graph.GraphBackedMetadataRepository$GraphToTypedInstanceMapper.mapGraphToTypedInstance(GraphBackedMetadataRepository.java:882)
>         at org.apache.atlas.discovery.graph.DefaultGraphPersistenceStrategy.constructInstance(DefaultGraphPersistenceStrategy.java:149)
>         at org.apache.atlas.query.GremlinEvaluator$$anonfun$2.apply(GremlinEvaluator.scala:89)
>         at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
>         at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
>         at scala.collection.Iterator$class.foreach(Iterator.scala:727)
>         at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
>         at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
>         at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
>         at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
>         at scala.collection.AbstractTraversable.map(Traversable.scala:105)
>         at org.apache.atlas.query.GremlinEvaluator.evaluate(GremlinEvaluator.scala:87)
>         at org.apache.atlas.discovery.graph.GraphBackedDiscoveryService.evaluate(GraphBackedDiscoveryService.java:150)
>         at org.apache.atlas.discovery.graph.GraphBackedDiscoveryService.evaluate(GraphBackedDiscoveryService.java:135)
>         ... 50 more
> And in the hive hiveserver2.log we see 
> 2015-12-15 13:29:06,217 INFO  [Atlas Logger 1]: hook.HiveHook (HiveHook.java:run(168))
- Atlas hook failed
> org.apache.atlas.AtlasServiceException: Metadata service API SEARCH_DSL failed with status
400(Bad Request) Response Body ({"error":"Invalid expression : hive_db where name = 'tsz'
and clusterName = 'primary'","stackTrace":"org.apache.atlas.discovery.DiscoveryException:
Invalid expression : hive_db where name = 'tsz' and clusterName = 'primary'\n\tat org.apache.atlas.discovery.graph.GraphBackedDiscoveryService.evaluate(GraphBackedDiscoveryService.java:140)\n\tat
org.apache.atlas.discovery.graph.GraphBackedDiscoveryService.searchByDSL(GraphBackedDiscoveryService.java:124)\n\tat
org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:41)\n\tat
org.apache.atlas.web.resources.MetadataDiscoveryResource.searchUsingQueryDSL(MetadataDiscoveryResource.java:128)\n\tat
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)\n\tat
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat
java.lang.reflect.Method.invoke(Method.java:606)\n\tat com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)\n\tat
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)\n\tat
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)\n\tat
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)\n\tat
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)\n\tat
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)\n\tat
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)\n\tat
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)\n\tat
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)\n\tat
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)\n\tat
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)\n\tat
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)\n\tat
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)\n\tat com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)\n\tat
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)\n\tat
javax.servlet.http.HttpServlet.service(HttpServlet.java:820)\n\tat com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:287)\n\tat
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:277)\n\tat com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:182)\n\tat
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)\n\tat
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)\n\tat
org.apache.atlas.web.filters.AuditFilter.doFilter(AuditFilter.java:67)\n\tat com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)\n\tat
com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:119)\n\tat
com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:133)\n\tat com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:130)\n\tat
com.google.inject.servlet.GuiceFilter$Context.call(GuiceFilter.java:203)\n\tat com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:130)\n\tat
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)\n\tat
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)\n\tat org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)\n\tat
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)\n\tat org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)\n\tat
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)\n\tat org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)\n\tat
org.mortbay.jetty.Server.handle(Server.java:326)\n\tat org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)\n\tat
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)\n\tat
org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)\n\tat org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)\n\tat
org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)\n\tat org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)\n\tat
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)\nCaused by:
java.lang.NullPointerException\n\tat org.apache.atlas.repository.graph.GraphBackedMetadataRepository$GraphToTypedInstanceMapper.mapGraphToTypedInstance(GraphBackedMetadataRepository.java:882)\n\tat
org.apache.atlas.discovery.graph.DefaultGraphPersistenceStrategy.constructInstance(DefaultGraphPersistenceStrategy.java:149)\n\tat
org.apache.atlas.query.GremlinEvaluator$$anonfun$2.apply(GremlinEvaluator.scala:89)\n\tat
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)\n\tat scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)\n\tat
scala.collection.Iterator$class.foreach(Iterator.scala:727)\n\tat scala.collection.AbstractIterator.foreach(Iterator.scala:1157)\n\tat
scala.collection.IterableLike$class.foreach(IterableLike.scala:72)\n\tat scala.collection.AbstractIterable.foreach(Iterable.scala:54)\n\tat
scala.collection.TraversableLike$class.map(TraversableLike.scala:244)\n\tat scala.collection.AbstractTraversable.map(Traversable.scala:105)\n\tat
org.apache.atlas.query.GremlinEvaluator.evaluate(GremlinEvaluator.scala:87)\n\tat org.apache.atlas.discovery.graph.GraphBackedDiscoveryService.evaluate(GraphBackedDiscoveryService.java:150)\n\tat
org.apache.atlas.discovery.graph.GraphBackedDiscoveryService.evaluate(GraphBackedDiscoveryService.java:135)\n\t...
50 more\n"})
>         at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:365)
>         at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:346)
>         at org.apache.atlas.AtlasClient.searchByDSL(AtlasClient.java:276)
>         at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.getEntityReferenceFromDSL(HiveMetaStoreBridge.java:188)
>         at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.getDatabaseReference(HiveMetaStoreBridge.java:169)
>         at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.registerDatabase(HiveMetaStoreBridge.java:103)
>         at org.apache.atlas.hive.hook.HiveHook.handleCreateDB(HiveHook.java:267)
>         at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:189)
>         at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:54)
>         at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:166)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message