hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11758) Add options to filter out too much granular tracing spans
Date Fri, 27 Mar 2015 18:52:53 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14384371#comment-14384371
] 

Colin Patrick McCabe commented on HADOOP-11758:
-----------------------------------------------

I also wonder if we can do tracing at a level slightly above writeChunk.  writeChunk operates
at the level of 512-byte chunks, but writes are often larger than that.

If you look here:
{code}
  private void writeChecksumChunks(byte b[], int off, int len)
  throws IOException {
    sum.calculateChunkedSums(b, off, len, checksum, 0);
    for (int i = 0; i < len; i += sum.getBytesPerChecksum()) {
      int chunkLen = Math.min(sum.getBytesPerChecksum(), len - i);
      int ckOffset = i / sum.getBytesPerChecksum() * getChecksumSize();
      writeChunk(b, off + i, chunkLen, checksum, ckOffset, getChecksumSize());
    }
  }
{code}
you can see that if we do a 4 kilobyte read, writeChunk will get called 8 times.  But really
it would be better just to have one span representing the entire 4k write.

> Add options to filter out too much granular tracing spans
> ---------------------------------------------------------
>
>                 Key: HADOOP-11758
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11758
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: tracing
>            Reporter: Masatake Iwasaki
>            Assignee: Masatake Iwasaki
>            Priority: Minor
>         Attachments: testWriteTraceHooks.html
>
>
> in order to avoid queue in span receiver spills



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message