hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth Jayachandran (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-23110) Prevent NPE in ReExecDriver if the processing is aborted
Date Wed, 01 Apr 2020 08:22:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-23110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17072509#comment-17072509
] 

Prasanth Jayachandran commented on HIVE-23110:
----------------------------------------------

I have partial logs
{code:java}
hiveserver2 <14>1 2020-03-31T20:52:24.702Z hiveserver2-0.hiveserver2-service.compute-1585643974-lwrg.svc.cluster.local
hiveserver2 1 6ba03ff1-251f-4878-81ea-1ba72d36c465 [mdc@18060 class="ql.Driver" level="INFO"
operationLogLevel="EXECUTION" queryId="hive_20200331205007_6397e486-03a9-41ec-a56b-e0c4ff1ff26e"
sessionId="94e0ab1a-e5ca-4237-9713-235b5dd2559a" thread="HiveServer2-Background-Pool: Thread-74"]
Executing command(queryId=hive_20200331205007_6397e486-03a9-41ec-a56b-e0c4ff1ff26e) has been
interrupted after 133.75 seconds
hiveserver2 <14>1 2020-03-31T20:52:24.702Z hiveserver2-0.hiveserver2-service.compute-1585643974-lwrg.svc.cluster.local
hiveserver2 1 6ba03ff1-251f-4878-81ea-1ba72d36c465 [mdc@18060 class="ql.Driver" level="INFO"
operationLogLevel="EXECUTION" queryId="hive_20200331205007_6397e486-03a9-41ec-a56b-e0c4ff1ff26e"
sessionId="94e0ab1a-e5ca-4237-9713-235b5dd2559a" thread="HiveServer2-Background-Pool: Thread-74"]
OK
hiveserver2 <15>1 2020-03-31T20:52:24.702Z hiveserver2-0.hiveserver2-service.compute-1585643974-lwrg.svc.cluster.local
hiveserver2 1 6ba03ff1-251f-4878-81ea-1ba72d36c465 [mdc@18060 class="log.PerfLogger" level="DEBUG"
operationLogLevel="EXECUTION" queryId="hive_20200331205007_6397e486-03a9-41ec-a56b-e0c4ff1ff26e"
sessionId="94e0ab1a-e5ca-4237-9713-235b5dd2559a" thread="HiveServer2-Background-Pool: Thread-74"]
<PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
hiveserver2 <14>1 2020-03-31T20:52:24.711Z hiveserver2-0.hiveserver2-service.compute-1585643974-lwrg.svc.cluster.local
hiveserver2 1 6ba03ff1-251f-4878-81ea-1ba72d36c465 [mdc@18060 class="common.LogUtils" level="INFO"
thread="HiveServer2-Background-Pool: Thread-74"] Unregistered logging context.
hiveserver2 <14>1 2020-03-31T20:52:24.702Z hiveserver2-0.hiveserver2-service.compute-1585643974-lwrg.svc.cluster.local
hiveserver2 1 6ba03ff1-251f-4878-81ea-1ba72d36c465 [mdc@18060 class="lockmgr.DbLockManager"
level="INFO" operationLogLevel="EXECUTION" queryId="hive_20200331205007_6397e486-03a9-41ec-a56b-e0c4ff1ff26e"
sessionId="94e0ab1a-e5ca-4237-9713-235b5dd2559a" thread="HiveServer2-Background-Pool: Thread-74"]
releaseLocks: 
hiveserver2 <15>1 2020-03-31T20:52:24.703Z hiveserver2-0.hiveserver2-service.compute-1585643974-lwrg.svc.cluster.local
hiveserver2 1 6ba03ff1-251f-4878-81ea-1ba72d36c465 [mdc@18060 class="log.PerfLogger" level="DEBUG"
operationLogLevel="EXECUTION" queryId="hive_20200331205007_6397e486-03a9-41ec-a56b-e0c4ff1ff26e"
sessionId="94e0ab1a-e5ca-4237-9713-235b5dd2559a" thread="HiveServer2-Background-Pool: Thread-74"]
</PERFLOG method=releaseLocks start=1585687944702 end=1585687944703 duration=1 from=org.apache.hadoop.hive.ql.Driver>
hiveserver2 <11>1 2020-03-31T20:52:24.711Z hiveserver2-0.hiveserver2-service.compute-1585643974-lwrg.svc.cluster.local
hiveserver2 1 6ba03ff1-251f-4878-81ea-1ba72d36c465 [mdc@18060 class="operation.Operation"
level="ERROR" operationLogLevel="EXECUTION" queryId="hive_20200331205007_6397e486-03a9-41ec-a56b-e0c4ff1ff26e"
sessionId="94e0ab1a-e5ca-4237-9713-235b5dd2559a" thread="HiveServer2-Background-Pool: Thread-74"]
Error running hive query: 
org.apache.hive.service.cli.HiveSQLException: Illegal Operation state transition from CANCELED
to FINISHED
    at org.apache.hive.service.cli.OperationState.validateTransition(OperationState.java:97)
    at org.apache.hive.service.cli.OperationState.validateTransition(OperationState.java:103)
    at org.apache.hive.service.cli.operation.Operation.setState(Operation.java:161)
    at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:248)
    at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:87)
    at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:322)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
    at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:340)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
hiveserver2 2020-03-31 20:52:24,710 Log4j2-TF-1-AsyncLogger[AsyncContext@18b4aac2]-1 ERROR
/tmp/hive/operation_logs/94e0ab1a-e5ca-4237-9713-235b5dd2559a/hive_20200331205007_6397e486-03a9-41ec-a56b-e0c4ff1ff26e
was closed recently. {code}
 

The failure hooks are running. I can see it running on user interrupt and it is expected behaviour.
One of my hook wasn't running before on interrupt but that was an issue in my hook. 

> Prevent NPE in ReExecDriver if the processing is aborted
> --------------------------------------------------------
>
>                 Key: HIVE-23110
>                 URL: https://issues.apache.org/jira/browse/HIVE-23110
>             Project: Hive
>          Issue Type: Bug
>          Components: HiveServer2
>            Reporter: Miklos Gergely
>            Assignee: Zoltan Haindrich
>            Priority: Major
>         Attachments: HIVE-23110.01.patch
>
>
> In case of abort the context would be null, and thus the planMapper can not be obtained
from it.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message