tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harish Jaiprakash (JIRA)" <j...@apache.org>
Subject [jira] [Created] (TEZ-3673) Allocate smaller buffers in UnorderedPartitionedKVWriter
Date Tue, 28 Mar 2017 14:51:42 GMT
Harish Jaiprakash created TEZ-3673:
--------------------------------------

             Summary: Allocate smaller buffers in UnorderedPartitionedKVWriter
                 Key: TEZ-3673
                 URL: https://issues.apache.org/jira/browse/TEZ-3673
             Project: Apache Tez
          Issue Type: Improvement
            Reporter: Harish Jaiprakash
            Assignee: Harish Jaiprakash


UnorderedPartitionedKVWriter allocates in bigger chunks. It may or may not get filled up.
In PipelinedSorter, we start off with 32MB chunks. But UnorderedPartitionedKVWriter can be
worse as it allocates bigger blocks. Need to revisit this allocation.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message