hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13438) Optimize IPC server protobuf decoding
Date Thu, 05 Jan 2017 19:25:58 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802271#comment-15802271

Andrew Wang commented on HADOOP-13438:

I'd like to plug the [versions|https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-versions/]
script that I'm using to reconcile trunk with the 3.0.0-alpha2 fix version. It automates the
diffing, and also lets you fixup commit message typos via a side metadata file.

I also did similar reconciliation work for alpha1, and [made a list of typos|https://github.com/umbrant/versions/tree/822f5c0685a5d87a9eaa8abca035514f4f6be3f0/trunk_as_of_05ede003868871addc30162e9707c3dc14ed6b7a]
that likely apply to 2.8.0 too.

> Optimize IPC server protobuf decoding
> -------------------------------------
>                 Key: HADOOP-13438
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13438
>             Project: Hadoop Common
>          Issue Type: Sub-task
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>             Fix For: 2.8.0, 3.0.0-alpha1
>         Attachments: HADOOP-13438.patch, HADOOP-13438.patch.1
> The current use of the protobuf API uses an expensive code path.  The builder uses the
parser to instantiate a message, then copies the message into the builder.  The parser is
creating multi-layered internally buffering streams that cause excessive byte[] allocations.
> Using the parser directly with a coded input stream backed by the byte[] from the wire
will take a fast-path straight to the pb message's ctor.  Substantially less garbage is generated.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message