drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andy Pernsteiner (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-3270) Modify HbaseGroupScan to allow multiple fragments per region
Date Wed, 10 Jun 2015 15:57:00 GMT
Andy Pernsteiner created DRILL-3270:

             Summary: Modify HbaseGroupScan to allow multiple fragments per region
                 Key: DRILL-3270
                 URL: https://issues.apache.org/jira/browse/DRILL-3270
             Project: Apache Drill
          Issue Type: Improvement
          Components: Query Planning & Optimization, Storage - HBase
    Affects Versions: 1.0.0
            Reporter: Andy Pernsteiner
            Assignee: Jinfeng Ni
            Priority: Minor

When performing a full HBASE or MapR-DB table scan using drill, it is observed within the
resulting query profile that only one minor fragment is assigned per region, regardless of
the size of the region.  In the case of extremely large regions, especially if there are regions
of mismatching sizes, this can result in poor performance and a low degree of parallelism.

One possible option (mentioned by sphillips) is to lazily compute the splits by assuming that
the keys within a given region were evenly distributed (not perfect, but better than nothing),
and perhaps have a 'max-frags-per-region' setting.

This message was sent by Atlassian JIRA

View raw message