drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Rogers (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-5104) Foreman sets external sort memory allocation even for a physical plan
Date Sun, 04 Dec 2016 07:54:58 GMT
Paul Rogers created DRILL-5104:
----------------------------------

             Summary: Foreman sets external sort memory allocation even for a physical plan
                 Key: DRILL-5104
                 URL: https://issues.apache.org/jira/browse/DRILL-5104
             Project: Apache Drill
          Issue Type: Bug
    Affects Versions: 1.8.0
            Reporter: Paul Rogers
            Assignee: Paul Rogers


Consider the (disabled) unit test {{TestSimpleExternalSort.outOfMemoryExternalSort}} which
uses the physical plan {{xsort/oom_sort_test.json}} that contains a setting for the amount
of memory to allocate:

{code}
       {
            ...
            pop:"external-sort",
            ...
            initialAllocation: 1000000,
            maxAllocation: 30000000
        },
{code}

When run, the amount of memory is set to 715827882. The reason is that code was added to {{Foreman}}
to compute the memory to allocate to the external sort:

{code}
  private void runPhysicalPlan(final PhysicalPlan plan) throws ExecutionSetupException {
    validatePlan(plan);
    MemoryAllocationUtilities.setupSortMemoryAllocations(plan, queryContext);
{code}

The problem is that a physical plan should execute as provided to enable detailed testing.

To solve this problem, move the sort memory setup to the path taken by SQL queries, but not
via physical plans.

This change is necessary to re-enable the previously-disabled external sort tests.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message