drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Rogers (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (DRILL-5026) ExternalSortBatch uses two memory allocators; one will do
Date Fri, 17 Feb 2017 22:23:44 GMT

     [ https://issues.apache.org/jira/browse/DRILL-5026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Paul Rogers resolved DRILL-5026.
    Resolution: Fixed

> ExternalSortBatch uses two memory allocators; one will do
> ---------------------------------------------------------
>                 Key: DRILL-5026
>                 URL: https://issues.apache.org/jira/browse/DRILL-5026
>             Project: Apache Drill
>          Issue Type: Sub-task
>    Affects Versions: 1.8.0
>            Reporter: Paul Rogers
>            Assignee: Paul Rogers
>            Priority: Minor
> The {{ExternalSortBatch}} (ESB) operator performs a sort while spilling to disk to stay
within a defined memory budget.
> ESB uses two memory allocators:
> * The operator allocator for most operations,
> * The copier allocator (child of the operator allocator) for merge operations.
> The copier allocator is used only when merging prior to spill.
> However, since the ESB must manage within a single memory budget, the code is made more
complex by using two allocators, and keeping track of the memory used in both.
> Better to use a single allocator. When checking for the need to spill, the code already
checks if enough memory is available for an in-memory sort, if needed. Similarly, the code
should also check if sufficient memory is available for the copy step when needed.

This message was sent by Atlassian JIRA

View raw message