hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiroshi Ikeda (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10278) Refactor to make CallQueue pluggable
Date Mon, 17 Feb 2014 05:17:20 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13902965#comment-13902965
] 

Hiroshi Ikeda commented on HADOOP-10278:
----------------------------------------

Although ReadWriteLock is much more complex than simple locks and is not prefered to enclose
trival logic, I suspect it is not severe overhead compared to LinkedBlockingQueue.

Anybody seems not to worry about using LinkedBlockingQueue, but LinkedBlockingQueue separates
its lock into only two locks, put side and get side, so that congesion in each side causes
context switches. Moreover the separation of the lock is ineffective when there are almost
always empty in the queue, and collision between put and get causes context switches.

By the way, about the patch, nobody can eliminate the possibility that a blocked thread at
LinkedBlockingQueue.put() cannot wake up in 1 seconds when another thread drains. At least
you should check the reference after put(), such as
{code}
void put(e) {
  q = queue.get();
  q.put(e);
  while((q2 = queue.get()) != q) {
    drain(q, q2);
    q = q2;
  }
}
{code}

(I also worry about the order of elements is not preserved in spite of the name "queue".)

Still, implementation of size() is invalid.

The atomic reference is not needed for the queue and volatile is enough if you only set and
get. Volatile variables may have much chance to be optimized by VM because volatile is within
the language specification. On the other hand, increment operation (++) is not atomic for
volatile variables, so some of the test classes should be changed.


> Refactor to make CallQueue pluggable
> ------------------------------------
>
>                 Key: HADOOP-10278
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10278
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: ipc
>            Reporter: Chris Li
>         Attachments: HADOOP-10278-atomicref-adapter.patch, HADOOP-10278-atomicref-adapter.patch,
HADOOP-10278-atomicref-rwlock.patch, HADOOP-10278-atomicref.patch, HADOOP-10278-atomicref.patch,
HADOOP-10278-atomicref.patch, HADOOP-10278-atomicref.patch, HADOOP-10278.patch, HADOOP-10278.patch
>
>
> * Refactor CallQueue into an interface, base, and default implementation that matches
today's behavior
> * Make the call queue impl configurable, keyed on port so that we minimize coupling



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message