trafodion-codereview mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From eowhadi <...@git.apache.org>
Subject [GitHub] incubator-trafodion pull request: [TRAFODION-1421] Implement paral...
Date Mon, 11 Apr 2016 17:35:06 GMT
GitHub user eowhadi opened a pull request:

    https://github.com/apache/incubator-trafodion/pull/422

    [TRAFODION-1421] Implement parallel scanner primitive

    new CQD: hbase_dop_parallel_scanner
    - default 0.0 = disabled
    - from 0.x to 1.0 -> the DOP will be dynamically assessed as a percentage of region
to scan. If 10 region, and cqd is .3 (30%), then 3 threads will be used...
    - from 2.0 and above, will take ceiling, and use it as fixed number of threads.
    Advantage of parallel scanner parallelism over ESP is on resource consumption. Parallel
scanner is just creating threads, while ESP are full proceses wth high memory consumption.
    For now, this feature is just a primitive that can be manually exercised. The next obvious
step is to make the compiler aware of it and pick it over ESP DOP when appropriate...

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/eowhadi/incubator-trafodion parallelScan

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-trafodion/pull/422.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #422
    
----
commit a7f9a628179fcd4077a566f558e653b1380eabc6
Author: Eric Owhadi <eric.owhadi@esgyn.com>
Date:   2016-04-11T16:59:23Z

    [TRAFODION-1421] Implement parallel scanner primitive
    new CQD: hbase_dop_parallel_scanner
    - default 0.0 = disabled
    - from 0.x to 1.0 -> the DOP will be dynamically assessed as a percentage of region
to scan. If 10 region, and cqd is .3 (30%), then 3 threads will be used...
    - from 2.0 and above, will take ceiling, and use it as fixed number of threads.
    Advantage of parallel scanner parallelism over ESP is on resource consumption. Parallel
scanner is just creating threads, while ESP are full proceses wth high memory consumption.
    For now, this feature is just a primitive that can be manually exercised. The next obvious
step is to make the compiler aware of it and pick it over ESP DOP when appropriate...

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message