incubator-hcatalog-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mariappan Asokan" <>
Subject Review Request 26536: Compress serialized bytes for all objects before encoding.
Date Fri, 10 Oct 2014 01:52:03 GMT

This is an automatically generated e-mail. To reply, visit:

Review request for hcatalog and hive.

Repository: hive-git


Currently all objects shipped via Hadoop Configuration are not compressed.  Compression is
limited to the data member "partitions" in InputJobInfo.  This patch will compress all serialized
objects.  This will cut down the traffic in the cluster when Configuration object is shipped
to all compute nodes.  Also, there is a 5 MB size limit on the Configuration object.  Compression
will make this limiit hardly reachable.


  hcatalog/core/src/main/java/org/apache/hive/hcatalog/common/ 4fdb5c9 
  hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/ 360e77b



No new test required.  All existing tests should pass.


Mariappan Asokan

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message