hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "kumar ravi (Jira)" <j...@apache.org>
Subject [jira] [Comment Edited] (HIVE-23412) Filter applied in the "LEFT JOIN" condition is pulling a null batch on last reducer (map join vectorizer operator) and throwing exception as "Hive Runtime Error while processing vector batch" at the map join during the query execution.
Date Fri, 08 May 2020 15:52:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-23412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17102646#comment-17102646
] 

kumar ravi edited comment on HIVE-23412 at 5/8/20, 3:51 PM:
------------------------------------------------------------

[~pgaref]: Issue is with Hive 3.1.0. Attached masked query with the JIRA.


was (Author: kumar.ravi):
[~pgaref]: Issue is with Hive 3.1.0. We cant share the query, Sorry for that.

> Filter applied in the "LEFT JOIN" condition is pulling a null batch on last reducer (map
join vectorizer operator) and throwing exception as "Hive Runtime Error while processing vector
batch" at the map join during the query execution.
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-23412
>                 URL: https://issues.apache.org/jira/browse/HIVE-23412
>             Project: Hive
>          Issue Type: Bug
>          Components: Vectorization
>    Affects Versions: 3.1.0
>            Reporter: kumar ravi
>            Priority: Major
>
> Filter applied in the "LEFT JOIN" condition is pulling a null batch on last reducer (map
join vectorizer operator) and throwing exception as "Hive Runtime Error while processing vector
batch" at the map join during the query execution.
> {{ERROR : FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask.
Vertex re-running, vertexName=Map 2, vertexId=vertex_1588769098286_8402_1_00Vertex re-running,
vertexName=Map 1, vertexId=vertex_1588769098286_8402_1_02Vertex re-running, vertexName=Map
4, vertexId=vertex_1588769098286_8402_1_04Vertex failed, vertexName=Reducer 6, vertexId=vertex_1588769098286_8402_1_05,
diagnostics=[Task failed, taskId=task_1588769098286_8402_1_05_000604, diagnostics=[TaskAttempt
0 failed, info=[Error: Error while running task ( failure ) : attempt_1588769098286_8402_1_05_000604_0:java.lang.RuntimeException:
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing vector batch (tag=3) (vectorizedVertexNum 5)}}{{ at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)}}{{
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)}}{{ at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)}}{{
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)}}{{
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)}}{{
at java.security.AccessController.doPrivileged(Native Method)}}{{ at javax.security.auth.Subject.doAs(Subject.java:422)}}{{
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)}}{{
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)}}{{
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)}}{{
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)}}{{ at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)}}{{
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)}}{{
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)}}{{
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}{{ at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}{{ at java.lang.Thread.run(Thread.java:748)}}{{Caused
by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing vector batch (tag=3) (vectorizedVertexNum 5)}}{{ at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecordVector(ReduceRecordSource.java:401)}}{{
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:249)}}{{
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:318)}}{{
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)}}{{
... 16 more}}{{Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error
while processing vector batch (tag=3) (vectorizedVertexNum 5)}}{{ at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.processVectorGroup(ReduceRecordSource.java:503)}}{{
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecordVector(ReduceRecordSource.java:392)}}{{
... 19 more}}{{Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unexpected exception
from VectorMapJoinOperator : null}}{{ at org.apache.hadoop.hive.ql.exec.MapJoinOperator.process(MapJoinOperator.java:539)}}{{
at org.apache.hadoop.hive.ql.exec.vector.VectorMapJoinOperator.process(VectorMapJoinOperator.java:235)}}{{
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.processVectorGroup(ReduceRecordSource.java:490)}}{{
... 20 more}}{{Caused by: java.lang.NullPointerException}}{{ at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.getFilterTag(CommonJoinOperator.java:802)}}{{
at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:600)}}{{
at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genJoinObject(CommonJoinOperator.java:533)}}{{
at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:936)}}{{
at org.apache.hadoop.hive.ql.exec.MapJoinOperator.process(MapJoinOperator.java:526)}}{{ ...
22 more}}{{], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) : attempt_1588769098286_8402_1_05_000604_1:java.lang.RuntimeException:
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing vector batch (tag=3) (vectorizedVertexNum 5)}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message