hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Dere (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-22967) Support hive.reloadable.aux.jars.path for Hive on Tez
Date Fri, 08 May 2020 16:22:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-22967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17102709#comment-17102709
] 

Jason Dere commented on HIVE-22967:
-----------------------------------

I think this makes sense. +1

> Support hive.reloadable.aux.jars.path for Hive on Tez
> -----------------------------------------------------
>
>                 Key: HIVE-22967
>                 URL: https://issues.apache.org/jira/browse/HIVE-22967
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 3.1.2, 2.3.6
>            Reporter: Toshihiko Uchida
>            Assignee: Toshihiko Uchida
>            Priority: Minor
>         Attachments: HIVE-22967.1.patch, HIVE-22967.2.patch
>
>
> The jars in hive.reloadable.aux.jars.path are not localized in Tez containers.
> As a result, any query utilizing those reloadable jars fails for Hive on Tez due to ClassNotFoundException.
> {code}
> Error: Error while processing statement: FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1578856704640_0087_1_00,
diagnostics=[Task failed, taskId=task_1578856704640_0087_1_00_000001, diagnostics=[TaskAttempt
0 failed, info=[Error: Error while running task ( failure) : attempt_1578856704640_0087_1_00_000001_0:java.lang.RuntimeException:
java.lang.RuntimeException: Map operator initialization failed
>         at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
>         at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
>         at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
>         at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
>         at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:422)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
>         at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
>         at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
>         at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
>         at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
>         at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
>         at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>         at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.RuntimeException: Map operator initialization failed
>         at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:354)
>         at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:266)
>         ... 16 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException:
java.lang.ClassNotFoundException: com.example.hive.udf.Lower
>         at org.apache.hadoop.hive.ql.exec.FilterOperator.initializeOp(FilterOperator.java:71)
>         at org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.initializeOp(VectorFilterOperator.java:83)
>         at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
>         at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:573)
>         at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:525)
>         at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:386)
>         at org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.initializeMapOperator(VectorMapOperator.java:591)
>         at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:317)
>         ... 17 more
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: com.example.hive.udf.Lower
>         at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.getUdfClass(GenericUDFBridge.java:134)
>         at org.apache.hadoop.hive.ql.exec.FunctionRegistry.isStateful(FunctionRegistry.java:1492)
>         at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.<init>(ExprNodeGenericFuncEvaluator.java:111)
>         at org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorFactory.get(ExprNodeEvaluatorFactory.java:58)
>         at org.apache.hadoop.hive.ql.exec.FilterOperator.initializeOp(FilterOperator.java:63)
>         ... 24 more
> Caused by: java.lang.ClassNotFoundException: com.example.hive.udf.Lower
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>         at java.lang.Class.forName0(Native Method)
>         at java.lang.Class.forName(Class.java:348)
>         at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.getUdfClassInternal(GenericUDFBridge.java:142)
>         at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.getUdfClass(GenericUDFBridge.java:132)
>         ... 28 more
> ], TaskAttempt 1 failed, (omitted)
> {code}
> HIVE-14037 and HIVE-14142 resolved the similar issues for MapReduce and Hive on Spark,
respectively, in the past.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message