From issues-return-198059-apmail-hive-issues-archive=hive.apache.org@hive.apache.org Tue Aug 25 17:46:02 2020 Return-Path: X-Original-To: apmail-hive-issues-archive@locus.apache.org Delivered-To: apmail-hive-issues-archive@locus.apache.org Received: from mailroute1-lw-us.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by minotaur.apache.org (Postfix) with ESMTP id 4449719642 for ; Tue, 25 Aug 2020 17:46:02 +0000 (UTC) Received: from mail.apache.org (localhost [127.0.0.1]) by mailroute1-lw-us.apache.org (ASF Mail Server at mailroute1-lw-us.apache.org) with SMTP id DE6FB124764 for ; Tue, 25 Aug 2020 17:46:01 +0000 (UTC) Received: (qmail 70955 invoked by uid 500); 25 Aug 2020 17:46:01 -0000 Delivered-To: apmail-hive-issues-archive@hive.apache.org Received: (qmail 70931 invoked by uid 500); 25 Aug 2020 17:46:01 -0000 Mailing-List: contact issues-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list issues@hive.apache.org Received: (qmail 70917 invoked by uid 99); 25 Aug 2020 17:46:01 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 25 Aug 2020 17:46:01 +0000 Received: from jira-he-de.apache.org (static.172.67.40.188.clients.your-server.de [188.40.67.172]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id B4FE84078A for ; Tue, 25 Aug 2020 17:46:00 +0000 (UTC) Received: from jira-he-de.apache.org (localhost.localdomain [127.0.0.1]) by jira-he-de.apache.org (ASF Mail Server at jira-he-de.apache.org) with ESMTP id 3A003780105 for ; Tue, 25 Aug 2020 17:46:00 +0000 (UTC) Date: Tue, 25 Aug 2020 17:46:00 +0000 (UTC) From: "ASF GitHub Bot (Jira)" To: issues@hive.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Work logged] (HIVE-24061) Improve llap task scheduling for better cache hit rate MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-24061?focusedWorklogId=3D= 474425&page=3Dcom.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpa= nel#worklog-474425 ] ASF GitHub Bot logged work on HIVE-24061: ----------------------------------------- Author: ASF GitHub Bot Created on: 25/Aug/20 17:45 Start Date: 25/Aug/20 17:45 Worklog Time Spent: 10m=20 Work Description: prasanthj commented on a change in pull request #14= 31: URL: https://github.com/apache/hive/pull/1431#discussion_r476626802 ########## File path: llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTas= kSchedulerService.java ########## @@ -1487,6 +1492,14 @@ private SelectHostResult selectHost(TaskInfo request= ) { return SELECT_HOST_RESULT_DELAYED_RESOURCES; } =20 + // When all nodes are busy, reset locality delay + if (activeNodesWithFreeSlots.isEmpty()) { + isCapacityFull.set(true); + if (request.localityDelayTimeout > 0 && isRequestedHostPresent(req= uest)) { + request.resetLocalityDelayInfo(); + } + } Review comment: isCapacityFull can be set to false in else condition here? instead o= f trySchedulingPendingTasks.. just a minor readability improvement..=20 ########## File path: llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTas= kSchedulerService.java ########## @@ -1817,8 +1830,10 @@ protected void schedulePendingTasks() throws Interru= ptedException { Iterator taskIter =3D taskListAtPriority.iterator(); boolean scheduledAllAtPriority =3D true; while (taskIter.hasNext()) { - // TODO Optimization: Add a check to see if there's any capacity= available. No point in - // walking through all active nodes, if they don't have potentia= l capacity. + // Early exit where there are no slots available + if (isCapacityFull.get()) { Review comment: This can be outer if condition? only if cluster capacity is availabl= e run the task iterator.=20 ########## File path: llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTas= kSchedulerService.java ########## @@ -1390,7 +1395,7 @@ private SelectHostResult selectHost(TaskInfo request)= { boolean shouldDelayForLocality =3D request.shouldDelayForLocality(sc= hedulerAttemptTime); LOG.debug("ShouldDelayForLocality=3D{} for task=3D{} on hosts=3D{}",= shouldDelayForLocality, request.task, requestedHostsDebugStr); - if (requestedHosts !=3D null && requestedHosts.length > 0) { + if (!isRequestedHostPresent(request)) { Review comment: the condition seems to have flipped here. is this expected? ########## File path: llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTas= kSchedulerService.java ########## @@ -251,6 +251,7 @@ public void setError(Void v, Throwable t) { =20 private final Lock scheduleLock =3D new ReentrantLock(); private final Condition scheduleCondition =3D scheduleLock.newCondition(= ); + private final AtomicBoolean isCapacityFull =3D new AtomicBoolean(false); Review comment: nit: to differentiate node vs cluster, rename this to isClusterCapac= ityFull?=20 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 474425) Time Spent: 20m (was: 10m) > Improve llap task scheduling for better cache hit rate=20 > ------------------------------------------------------- > > Key: HIVE-24061 > URL: https://issues.apache.org/jira/browse/HIVE-24061 > Project: Hive > Issue Type: Improvement > Reporter: Rajesh Balamohan > Assignee: Rajesh Balamohan > Priority: Major > Labels: perfomance, pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > TaskInfo is initialized with the "requestTime and locality delay". When l= ots of vertices are in the same level, "taskInfo" details would be availabl= e upfront. By the time, it gets to scheduling, "requestTime + localityDelay= " won't be higher than current time. Due to this, it misses scheduling dela= y details and ends up choosing random node. This ends up missing cache hits= and reads data from remote storage. > E.g Observed this pattern in Q75 of tpcds. > Related lines of interest in scheduler: [https://github.com/apache/hive/b= lob/master/llap-tez/src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTas= kSchedulerService.java |https://github.com/apache/hive/blob/master/llap-tez= /src/java/org/apache/hadoop/hive/llap/tezplugins/LlapTaskSchedulerService.j= ava] > {code:java} > boolean shouldDelayForLocality =3D request.shouldDelayForLocality(sche= dulerAttemptTime); > .. > .. > boolean shouldDelayForLocality(long schedulerAttemptTime) { > return localityDelayTimeout > schedulerAttemptTime; > } > {code} > =C2=A0 > Ideally, "localityDelayTimeout" should be adjusted based on it's first sc= heduling opportunity. -- This message was sent by Atlassian Jira (v8.3.4#803005)