trafodion-codereview mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From selvaganesang <...@git.apache.org>
Subject [GitHub] incubator-trafodion pull request #1228: [TRAFODION-2733] Provide an improved...
Date Wed, 13 Sep 2017 00:14:10 GMT
Github user selvaganesang commented on a diff in the pull request:

    https://github.com/apache/incubator-trafodion/pull/1228#discussion_r138498166
  
    --- Diff: core/sql/executor/ex_exe_stmt_globals.h ---
    @@ -307,18 +307,38 @@ class ExExeStmtGlobals : public ex_globals
     
       inline NABoolean grabMemoryQuotaIfAvailable(ULng32 size)
       { 
    +    CliGlobals *cli_globals = GetCliGlobals();
    +    if (cli_globals->isEspProcess())
    +       return cli_globals->grabMemoryQuotaIfAvailable(size);
         if ( unusedBMOsMemoryQuota_ < size ) return FALSE;
         unusedBMOsMemoryQuota_ -= size ;
         return TRUE;
       }
     
    -  inline void resetMemoryQuota() { unusedBMOsMemoryQuota_ = 0 ; }
    +  inline void resetMemoryQuota() 
    +  {
    +    CliGlobals *cli_globals = GetCliGlobals();
    +    if (cli_globals->isEspProcess())
    +       return cli_globals->resetMemoryQuota();
    +    unusedBMOsMemoryQuota_ = 0 ; 
    +  }
    --- End diff --
    
    In case of master process the unused memory quota is yielded to others operators within
the statement and hence it is stored in statement globals. In case of ESP, the unused memory
quota to yielded to operators in other fragments within the process and hence it is stored
in the default context.  The part of the code deals with the unused memory quota.
    
     I am assuming that you want details about the quota assignment itself.  Based on this
assumption:
    
     Each open statement in the master would get its own quota.   If it is non-ESP plan, each
BMO operator will be capped to max BMO limit of 1200 MB as opposed to 10240 MB per node. If
there are more than one BMO operator in such a plan,  each BMO operator memory quota will
be assigned proportionally as per item 14 subjected to min and max limits.
    
    it is a ESP plan, the memory quota will be assigned as per item 14 subjected to min and
max limit. 
    Currently,  Trafodion won't assign the instances from a same fragment to a process. It
also won't assign the ESP to two different queries either from the same context or different
contexts.  Because TDB contains the memory quota assignment per instance, different instances
within the same process should work irrespective how the instances are assigned.
    
    Yielding the unused memory quota will work fine as long as the ESP is not shared for different
ESPs at the same time.  Currently the ESP is released only when the statement is deallocated.
 The   current plan is not to share ESPs when it is multi-threaded. It is just that all the
fragments and all the instances within the same fragment  from a query will be hosted in the
same process via different threads.
    
    Hence,
    1) Yes. Each open statement would get its own quota assignment
    2) All ESPs and BMO operator instances memory consumption from a query will be limited
by BMO_MEMORY_LIMIT_PER_NODE. However, memory consumption won't be exact amount as specified
by this CQD because the memory quota assignment is subjected to min and max limits.
    3) ESPs are not shared currently.  There is no plan to share in the multi-threaded ESP
environment also.
    4) . Each statement will have its own quota irrespective of the context to which its belongs


---

Mime
View raw message