hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Panagiotis Garefalakis (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-23412) Filter applied in the "LEFT JOIN" condition is pulling a null batch on last reducer (map join vectorizer operator) and throwing exception as "Hive Runtime Error while processing vector batch" at the map join during the query execution.
Date Fri, 08 May 2020 13:23:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-23412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17102586#comment-17102586
] 

Panagiotis Garefalakis commented on HIVE-23412:
-----------------------------------------------

Hey [~kumar.ravi] thanks for reporting this – in which version did you observe this behaviour?
Do you have a query that can reproduce this by any chance?

> Filter applied in the "LEFT JOIN" condition is pulling a null batch on last reducer (map
join vectorizer operator) and throwing exception as "Hive Runtime Error while processing vector
batch" at the map join during the query execution.
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-23412
>                 URL: https://issues.apache.org/jira/browse/HIVE-23412
>             Project: Hive
>          Issue Type: Bug
>            Reporter: kumar ravi
>            Priority: Major
>
> Filter applied in the "LEFT JOIN" condition is pulling a null batch on last reducer (map
join vectorizer operator) and throwing exception as "Hive Runtime Error while processing vector
batch" at the map join during the query execution.
> {{ERROR : FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask.
Vertex re-running, vertexName=Map 2, vertexId=vertex_1588769098286_8402_1_00Vertex re-running,
vertexName=Map 1, vertexId=vertex_1588769098286_8402_1_02Vertex re-running, vertexName=Map
4, vertexId=vertex_1588769098286_8402_1_04Vertex failed, vertexName=Reducer 6, vertexId=vertex_1588769098286_8402_1_05,
diagnostics=[Task failed, taskId=task_1588769098286_8402_1_05_000604, diagnostics=[TaskAttempt
0 failed, info=[Error: Error while running task ( failure ) : attempt_1588769098286_8402_1_05_000604_0:java.lang.RuntimeException:
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing vector batch (tag=3) (vectorizedVertexNum 5)}}{{ at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)}}{{
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)}}{{ at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)}}{{
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)}}{{
at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)}}{{
at java.security.AccessController.doPrivileged(Native Method)}}{{ at javax.security.auth.Subject.doAs(Subject.java:422)}}{{
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)}}{{
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)}}{{
at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)}}{{
at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)}}{{ at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)}}{{
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)}}{{
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)}}{{
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}{{ at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}{{ at java.lang.Thread.run(Thread.java:748)}}{{Caused
by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing vector batch (tag=3) (vectorizedVertexNum 5)}}{{ at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecordVector(ReduceRecordSource.java:401)}}{{
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:249)}}{{
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordProcessor.run(ReduceRecordProcessor.java:318)}}{{
at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)}}{{
... 16 more}}{{Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error
while processing vector batch (tag=3) (vectorizedVertexNum 5)}}{{ at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.processVectorGroup(ReduceRecordSource.java:503)}}{{
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecordVector(ReduceRecordSource.java:392)}}{{
... 19 more}}{{Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unexpected exception
from VectorMapJoinOperator : null}}{{ at org.apache.hadoop.hive.ql.exec.MapJoinOperator.process(MapJoinOperator.java:539)}}{{
at org.apache.hadoop.hive.ql.exec.vector.VectorMapJoinOperator.process(VectorMapJoinOperator.java:235)}}{{
at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.processVectorGroup(ReduceRecordSource.java:490)}}{{
... 20 more}}{{Caused by: java.lang.NullPointerException}}{{ at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.getFilterTag(CommonJoinOperator.java:802)}}{{
at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genObject(CommonJoinOperator.java:600)}}{{
at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genJoinObject(CommonJoinOperator.java:533)}}{{
at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:936)}}{{
at org.apache.hadoop.hive.ql.exec.MapJoinOperator.process(MapJoinOperator.java:526)}}{{ ...
22 more}}{{], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) : attempt_1588769098286_8402_1_05_000604_1:java.lang.RuntimeException:
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing vector batch (tag=3) (vectorizedVertexNum 5)}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message