drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacques Nadeau <jacques.dr...@gmail.com>
Subject Re: Expressions in the Logical Plan
Date Sun, 18 Nov 2012 23:32:14 GMT
This is great!  I will look more at this.  I'm skimmed some of the optiq
code as well the optiq-splunk implementation and the linq4j work.  Do you
have any docs that give a conceptual overview of the various components and
their interrelationship as well as what was built to pluggable versus what
was not? (I saw your preso that you did at the Drill meetup in Sept.)  I've
been imagining that Optiq becomes the core of the optimizer in Drill based
on abstract notions but don't yet have a good handle on what that really
means.

thanks,
Jacques

On Sun, Nov 18, 2012 at 1:02 PM, Julian Hyde <julianhyde@gmail.com> wrote:

> On Nov 17, 2012, at 1:13 PM, Jacques Nadeau <jacques.drill@gmail.com>
> wrote:
>
> > Let me give an answer that may help understanding.  The logical plan
> tries
> > to be as generic as possible.  As such, all types are basic types.  That
> > being said, we'll surely have a large selection of various types of
> > functions which could modify a value.  For example currency(integer)
> might
> > convert a number of pennies into a dollar amount 3500 > "$35.00".  The
> > query layer will be responsible for converting the particular query
> > language to a set of these various functions.  Then the optimizer decides
> > where to apply those functions.  Lastly, the execution layer runs them as
> > efficiently as possible givens it execution environment.  Given all of
> > this, it is highly likely (especially early on) that a new parser may
> also
> > introduce new functions that need to be supported at the execution layer.
>
> Sorry I've been quiet on the list recently.
>
> I would be delighted to help Drill use the Optiq framework. It has all of
> this stuff, is proven and extensible.
>
> Specially regarding expressions. Take a look at RexProgram. It models a
> collection of expressions required to implement a project ("select clause")
> and a filter. RexProgram is a convenient base for common-subexpression
> elimination, shortcut generation, and so forth.
>
> Some of the finer details: There is an array of expressions, topologically
> sorted such that each expression is based only on expressions earlier in
> the array. Expressions are either inputs, literals, or calls to functions;
> functions can have multiple arguments, but the arguments must be references
> to expressions earlier in the array. Expressions are very strongly typed;
> casting which might be implicit in the query has been made explicit, so
> arguments have exactly the type required by the function.  Although it is a
> java class, it would easily serialize to a form similar to a text form
> similar to the examples in this thread.
>
>
> https://github.com/julianhyde/optiq/blob/master/src/main/java/org/eigenbase/rex/RexProgram.java
>
> I've been busy on my book and other projects, and I remain too busy to
> take ownership for of a Drill component, but I would be happy to help
> someone else evaluate Optiq (or pieces of Optiq) for use in Drill.
>
> Julian

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message