hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12528) Avoid spinning in CallQueueManager.take()
Date Mon, 02 Nov 2015 14:55:27 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14985333#comment-14985333
] 

Daryn Sharp commented on HADOOP-12528:
--------------------------------------

How expensive is the timedout poll?  The poll did have an unintended positive side-effect
in the jvm metrics.  Threads blocked roughly means threads blocked on the namesystem lock.
 Threads timed-waiting roughly mean calls waiting on the callq.  We somewhat rely on this
when debugging.

> Avoid spinning in CallQueueManager.take()
> -----------------------------------------
>
>                 Key: HADOOP-12528
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12528
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: performance
>    Affects Versions: 2.7.1
>            Reporter: Staffan Friberg
>            Assignee: Staffan Friberg
>            Priority: Minor
>         Attachments: HADOOP-12528.001.patch, HADOOP-12528.002.patch
>
>
> When IPC threads (Server$Handler) does take() to get the next Call, the CallManager does
a poll instead of take() on the internal queue.
> This causes threads to wake up and unnecessarily waste some CPU and do extra allocation
as part of the internal await/signal mechanism each time the thread redoes poll().
> This patch uses take() on the queue instead of poll() which will keep thread in the await
state until work is available. Since threads will be blocked on the queue indefinitely the
swapping of queues requires a bit of extra work to make sure threads wake up and does take
on the new queue.
> Updated the test TestCallQueueManager.testSwapUnderContention() to ensure that no threads
get stuck on the old queue as part of swapping.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message