tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rajesh Balamohan (JIRA)" <j...@apache.org>
Subject [jira] [Created] (TEZ-3789) Consider avoiding buffer copies in TezMerger when lots of unique keys are present in reducer side
Date Mon, 10 Jul 2017 04:33:00 GMT
Rajesh Balamohan created TEZ-3789:
-------------------------------------

             Summary: Consider avoiding buffer copies in TezMerger when lots of unique keys
are present in reducer side
                 Key: TEZ-3789
                 URL: https://issues.apache.org/jira/browse/TEZ-3789
             Project: Apache Tez
          Issue Type: Improvement
            Reporter: Rajesh Balamohan


Currently TezMerger stores the key details in memory. However, depending on the number of
records read, number of unique keys and the merger progress information, it should be possible
to deduce whether lots of unique keys are present and based on that buffer copies could be
avoided.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message