hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6949) Reduces RPC packet size for primitive arrays, especially long[], which is used at block reporting
Date Fri, 25 Feb 2011 18:48:21 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12999469#comment-12999469

Suresh Srinivas commented on HADOOP-6949:

> As I said in my comment in HDFS-1583, Java does not support heterogeneous arrays, see
ArrayStoreException for the reference. So we don't need to support such generic semantic.
We can assume the array is strictly homogeneous.

Assuming you are talking about labeling the array only for the type it carries, the approach
you are suggesting will not work for heartbeat response, where DatanodeCommand[] is the response,
with array elements carrying subtypes of DatanodeCommand, which should be labelled once per
array element.

Also a protocol could choose to give a response of Object[]. Not sure I understand the reference
you posted and how it is relevant to this discussion.

I prefer this issue focusing on primitive types and opening a separate jira for non-primitive
type, as I think it needs more discussion.

> Reduces RPC packet size for primitive arrays, especially long[], which is used at block
> -------------------------------------------------------------------------------------------------
>                 Key: HADOOP-6949
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6949
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: io
>            Reporter: Navis
>            Assignee: Matt Foley
>             Fix For: 0.23.0
>         Attachments: ObjectWritable.diff, arrayprim.patch, arrayprim_v4.patch
>   Original Estimate: 10m
>  Remaining Estimate: 10m
> Current implementation of oah.io.ObjectWritable marshals primitive array types as general
object array ; array type string + array length + (element type string + value)*n
> It would not be needed to specify each element types for primitive arrays.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message