incubator-hcatalog-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mariappan Asokan" <>
Subject Re: Review Request 26536: Compress serialized bytes for all objects before encoding.
Date Fri, 10 Oct 2014 16:35:58 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated Oct. 10, 2014, 4:35 p.m.)

Review request for hcatalog and hive.


Picked up the latest from the trunk.

Repository: hive-git


Currently all objects shipped via Hadoop Configuration are not compressed.  Compression is
limited to the data member "partitions" in InputJobInfo.  This patch will compress all serialized
objects.  This will cut down the traffic in the cluster when Configuration object is shipped
to all compute nodes.  Also, there is a 5 MB size limit on the Configuration object.  Compression
will make this limiit hardly reachable.

Diffs (updated)

  hcatalog/core/src/main/java/org/apache/hive/hcatalog/common/ 2f624df 
  hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/ 360e77b



No new test required.  All existing tests should pass.


Mariappan Asokan

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message