drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lisen Mu <imm...@gmail.com>
Subject Re: [jira] [Created] (DRILL-60) multiple query support
Date Mon, 13 May 2013 08:32:35 GMT
It's not a feature request for current stage of drill, however I would like
to contribute to it as long as we have a working local query. It's very
useful for us, and I thinks it's also an interesting problem.


Thanks,
Lisen



On Mon, May 13, 2013 at 4:21 PM, Lisen Mu (JIRA) <jira@apache.org> wrote:

> Lisen Mu created DRILL-60:
> -----------------------------
>
>              Summary: multiple query support
>                  Key: DRILL-60
>                  URL: https://issues.apache.org/jira/browse/DRILL-60
>              Project: Apache Drill
>           Issue Type: New Feature
>             Reporter: Lisen Mu
>
>
> parse/optimize several SQL(or other QL)s into one logical plan and execute
> them once.
>
> * use case
> If a report generating system is used as a frontend system of drill
> instead of a command line, There would be a set of similar SQL to execute
> to generate a report(consider each day's DAU statistics with geoip
> distribution(groupby) for 7 continuous days). If processed individually,
> plans for SQLs in this kind of set are similar(scan pv log in these days)
> or same(scan GeoIP table) in majority. If processed together, duplicated
> scan(GeoIP) could be eliminated to reduce I/O, or similar scan(pv) could be
> merged to achieve better throughput, and provides chances of other
> intermediate result reuse.
>
> * possibility
> Query in a QL is represented as a tree. Drill models logical plan and
> physical plan as DAG, which is more expressive than tree. More precisely, a
> set of trees(with common nodes or common edges) can be union-ed into a dag.
> So, the current drill logical/physical plan model supports multiple query
> already. Each sql would have a corresponding StoreOP as its result.
>
> * problems
> To do so, physical operators must support multiple iteration over its
> output. That would have side effect on mem management.
>
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message