hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9845) Update protobuf to 2.5 from 2.4.x
Date Mon, 09 Dec 2013 20:02:07 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843478#comment-13843478

Konstantin Shvachko commented on HADOOP-9845:

Mikhail, we can port it to 2.0, but there are things to consider.
I know a few binary distributions that are based on 2.0. 
And people may consider this as an incompatible change since it can affect particularly their
cluster management software.
Other than that it is just a matter of porting the patch.

> Update protobuf to 2.5 from 2.4.x
> ---------------------------------
>                 Key: HADOOP-9845
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9845
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: performance
>    Affects Versions: 2.0.5-alpha
>            Reporter: stack
>            Assignee: Alejandro Abdelnur
>            Priority: Blocker
>             Fix For: 2.1.0-beta
>         Attachments: HADOOP-9845.patch, HADOOP-9845.patch
> protobuf 2.5 is a bit faster with a new Parse to avoid a builder step and a few other
goodies that we'd like to take advantage of over in hbase especially now we are all pb all
the time.  Unfortunately the protoc generated files are no longer compatible w/ 2.4.1 generated
files.  Hadoop uses 2.4.1 pb.  This latter fact makes it so we cannot upgrade until hadoop
> This issue suggests hadoop2 move to protobuf 2.5.
> I can do the patch no prob. if there is interest.
> (When we upgraded our build broke with complaints like the below:
> {code}
> java.lang.UnsupportedOperationException: This is supposed to be overridden by subclasses.
> 	at com.google.protobuf.GeneratedMessage.getUnknownFields(GeneratedMessage.java:180)
> 	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$GetDatanodeReportRequestProto.getSerializedSize(ClientNamenodeProtocolProtos.java:21566)
> 	at com.google.protobuf.AbstractMessageLite.toByteString(AbstractMessageLite.java:49)
> 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.constructRpcRequest(ProtobufRpcEngine.java:149)
> 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:193)
> 	at com.sun.proxy.$Proxy14.getDatanodeReport(Unknown Source)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at java.lang.reflect.Method.invoke(Method.java:597)
> 	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:164)
> 	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:83)
> 	at com.sun.proxy.$Proxy14.getDatanodeReport(Unknown Source)
> 	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getDatanodeReport(ClientNamenodeProtocolTranslatorPB.java:488)
> 	at org.apache.hadoop.hdfs.DFSClient.datanodeReport(DFSClient.java:1887)
> 	at org.apache.hadoop.hdfs.MiniDFSCluster.waitActive(MiniDFSCluster.java:1798
> ...
> {code}
> More over in HBASE-8165 if interested.

This message was sent by Atlassian JIRA

View raw message