hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9688) Add globally unique request ID to RPC requests
Date Wed, 03 Jul 2013 04:34:23 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13698589#comment-13698589

Chris Nauroth commented on HADOOP-9688:

Thanks, Suresh.  Here are responses to the questions you posted.

If we can identify an instance of client uniquely, say clientID, then clientID + callID will
identify requests uniquely from a single client (based on callID), from two different clients
on the same machine (based on clientID) and from two different clients on different machines
(again based on clientID).

If I understand correctly, this change would generate the UUID once during client initialization,
and reuse that same UUID on every request (in combination with the existing call ID).  This
seems like a reasonable optimization.  Risk of collision seems highly unlikely.  (Call ID
is int, so a single client would need to generate 2^32 non-idempotent RPC calls within the
retry cache timeout before causing a collision.)

RPC request callID essentially has become a sequence number. Should we change the field name
to sequence number?

The phrase "sequence number" implies to me guaranteed in-order delivery and execution, like
the TCP sequence number.  I don't think our call ID is used in this way, so I think it's fine
to keep calling it a call ID.

Do we need to add requestID in response also? I do not see the need.

I also do not see the need.

For SaslRpcClient I am seeting the request ID to empty byte array, because retry is not expected
for this and this request will not be handled by RPC server implementation. Does this look

This seems fine, especially given that this code already uses a hard-coded call ID.  I'd appreciate
a second opinion though.

Also I want to call out attendion to - this change adds UUID for all RPC requests, irrespective
of if an RPC server implementation handles retransmits or not. Optimizing this out seems unnecessary.

Yes, agreed.

The patch itself is looking good, but I'll hold off on testing to see if you want to post
a new version converting to client ID instead of request ID.

    protected Call(RPC.RpcKind rpcKind, Writable param) {
      this.rpcKind = rpcKind;
      this.rpcRequest = param;
      synchronized (Client.this) {
        this.id = counter++;

I've been wondering if we should make a small optimization here by changing {{counter}} to
an {{AtomicInteger}} and avoiding lock contention on the {{Client}} instance.  This is unrelated
to your patch, so let me know if you prefer to handle it separately and I'll file a separate

> Add globally unique request ID to RPC requests
> ----------------------------------------------
>                 Key: HADOOP-9688
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9688
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: ipc
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>         Attachments: HADOOP-9688.patch
> This is a subtask in hadoop-common related to HDFS-4942 to add unique request ID to RPC

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message