hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Panagiotis Garefalakis (Jira)" <>
Subject [jira] [Commented] (HIVE-23443) LLAP speculative task pre-emption seems to be not working
Date Tue, 12 May 2020 10:32:00 GMT


Panagiotis Garefalakis commented on HIVE-23443:

Hey [~prasanth_j] the described logic does make sense to me – we should probably write
it down with capital letters somewhere in TaskExecutorService class :)
Anyway, left some comments in the PR let me know what you think

> LLAP speculative task pre-emption seems to be not working
> ---------------------------------------------------------
>                 Key: HIVE-23443
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>            Priority: Major
>         Attachments: HIVE-23443.1.patch
> I think after HIVE-23210 we are getting a stable sort order and it is causing pre-emption
to not work in certain cases.
> {code:java}
> "attempt_1589167813851_0000_119_01_000008_0 (hive_20200511055921_89598f09-19f1-4969-ab7a-82e2dd796273-119/Map
1, started at 2020-05-11 05:59:22, in preemption queue, can finish)", 
> "attempt_1589167813851_0008_84_01_000008_1 (hive_20200511055928_7ae29ca3-e67d-4d1f-b193-05651023b503-84/Map
1, started at 2020-05-11 06:00:23, in preemption queue, can finish)" {code}
> Scheduler only peek's at the pre-emption queue and looks at whether it is non-finishable. 
> []
> In the above case, all tasks are speculative but state change is not triggering pre-emption
queue re-ordering so peek() always returns canFinish task even though non-finishable tasks
are in the queue. 

This message was sent by Atlassian Jira

View raw message