hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9688) Add globally unique request ID to RPC requests
Date Wed, 03 Jul 2013 04:34:23 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13698589#comment-13698589
] 

Chris Nauroth commented on HADOOP-9688:
---------------------------------------

Thanks, Suresh.  Here are responses to the questions you posted.

{quote}
If we can identify an instance of client uniquely, say clientID, then clientID + callID will
identify requests uniquely from a single client (based on callID), from two different clients
on the same machine (based on clientID) and from two different clients on different machines
(again based on clientID).
{quote}

If I understand correctly, this change would generate the UUID once during client initialization,
and reuse that same UUID on every request (in combination with the existing call ID).  This
seems like a reasonable optimization.  Risk of collision seems highly unlikely.  (Call ID
is int, so a single client would need to generate 2^32 non-idempotent RPC calls within the
retry cache timeout before causing a collision.)

{quote}
RPC request callID essentially has become a sequence number. Should we change the field name
to sequence number?
{quote}

The phrase "sequence number" implies to me guaranteed in-order delivery and execution, like
the TCP sequence number.  I don't think our call ID is used in this way, so I think it's fine
to keep calling it a call ID.

{quote}
Do we need to add requestID in response also? I do not see the need.
{quote}

I also do not see the need.

{quote}
For SaslRpcClient I am seeting the request ID to empty byte array, because retry is not expected
for this and this request will not be handled by RPC server implementation. Does this look
right?
{quote}

This seems fine, especially given that this code already uses a hard-coded call ID.  I'd appreciate
a second opinion though.

{quote}
Also I want to call out attendion to - this change adds UUID for all RPC requests, irrespective
of if an RPC server implementation handles retransmits or not. Optimizing this out seems unnecessary.
{quote}

Yes, agreed.

The patch itself is looking good, but I'll hold off on testing to see if you want to post
a new version converting to client ID instead of request ID.

{code}
    protected Call(RPC.RpcKind rpcKind, Writable param) {
      this.rpcKind = rpcKind;
      this.rpcRequest = param;
      synchronized (Client.this) {
        this.id = counter++;
      }
{code}

I've been wondering if we should make a small optimization here by changing {{counter}} to
an {{AtomicInteger}} and avoiding lock contention on the {{Client}} instance.  This is unrelated
to your patch, so let me know if you prefer to handle it separately and I'll file a separate
jira.

                
> Add globally unique request ID to RPC requests
> ----------------------------------------------
>
>                 Key: HADOOP-9688
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9688
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: ipc
>            Reporter: Suresh Srinivas
>            Assignee: Suresh Srinivas
>         Attachments: HADOOP-9688.patch
>
>
> This is a subtask in hadoop-common related to HDFS-4942 to add unique request ID to RPC
requests.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message