incubator-hcatalog-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mariappan Asokan" <>
Subject Re: Review Request 26536: Compress serialized bytes for all objects before encoding.
Date Fri, 10 Oct 2014 16:02:56 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated Oct. 10, 2014, 4:02 p.m.)

Review request for hcatalog and hive.


Removed imports that are not needed anymore from

Repository: hive-git


Currently all objects shipped via Hadoop Configuration are not compressed.  Compression is
limited to the data member "partitions" in InputJobInfo.  This patch will compress all serialized
objects.  This will cut down the traffic in the cluster when Configuration object is shipped
to all compute nodes.  Also, there is a 5 MB size limit on the Configuration object.  Compression
will make this limiit hardly reachable.

Diffs (updated)

  hcatalog/core/src/main/java/org/apache/hive/hcatalog/common/ 4fdb5c9 
  hcatalog/core/src/main/java/org/apache/hive/hcatalog/mapreduce/ 360e77b



No new test required.  All existing tests should pass.


Mariappan Asokan

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message