hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Shelukhin (JIRA)" <>
Subject [jira] [Commented] (HIVE-11355) Hive on tez: memory manager for sort buffers (input/output) and operators
Date Fri, 07 Aug 2015 19:59:46 GMT


Sergey Shelukhin commented on HIVE-11355:

What is the magic with TEN_MB, should it be configurable?
remainingMemory = totalAvailableMemory / 2 nit - can be assigned in one statement in the line
Why is totalRequiredInputOutputMem required? Is Tez supposed to hold entire input and entire
output in memory?
if (remainingMemory < 0) { - log?

Should any of this be in explain (extended?) output?
Would any other q tests be affected by the addition of "Dummy Store" line to explain?

> Hive on tez: memory manager for sort buffers (input/output) and operators
> -------------------------------------------------------------------------
>                 Key: HIVE-11355
>                 URL:
>             Project: Hive
>          Issue Type: Improvement
>          Components: Tez
>    Affects Versions: 2.0.0
>            Reporter: Vikram Dixit K
>            Assignee: Vikram Dixit K
>         Attachments: HIVE-11355.1.patch, HIVE-11355.2.patch, HIVE-11355.3.patch
> We need to better manage the sort buffer allocations to ensure better performance. Also,
we need to provide configurations to certain operators to stay within memory limits.

This message was sent by Atlassian JIRA

View raw message