calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Milinda Pathirage <mpath...@umail.iu.edu>
Subject Re: Merging Project and Scan when there are RexCalls in projections
Date Thu, 19 Mar 2015 02:25:57 GMT
Hi Julian,

Thanks for the response.

Yes indeed, it looks like there will be several Samza specific push-downs
and RelNodes. I'm still trying to understand how conventions work and what
is there for Samza. But I'm sure we will need a Samza specific convention
as we grow.

Milinda


On Wed, Mar 18, 2015 at 4:53 PM, Julian Hyde <julian@hydromatic.net> wrote:

> ProjectableTable and ProjectableFilterableTable are compromises. They make
> it easy for someone to write a Calcite adapter for a data source, and
> perform the most common push-downs. 80% of the value for 20% of the effort.
>
> I thought it was worth pushing down filter expressions (e.g. x > 5 AND y
> is null) but I didn’t think it was worth pushing down projections. The
> ProjectableFilterableTable.scan API reflects this.
>
> If someone is an expert in Calcite, and has a data source that is capable
> of doing significant computation (e.g. covers a lot of the built-in SQL
> functions, user-defined functions, aggregation…) then they should build a
> “real” adapter. They will also need to write their own RelNodes, push-down
> rules, and probably their own convention. FWIW I think Samza is in that
> category.
>
> Julian
>
>
> On Mar 18, 2015, at 7:51 AM, Milinda Pathirage <mpathira@umail.iu.edu>
> wrote:
>
> > Hi Julian,
> >
> > I am trying to understand why Project and Scan merging is avoided when
> > there are projections which are RexCalls. Is this a purely implementation
> > related rule (such as separation of concerns or keeping scan as simple as
> > possible) or is there any theoretical background for this.
> >
> > I am asking this because I discovered that Project#getMapping returns
> null
> > when there are RexCalls in the projection which prevent
> ProjectableScanRule
> > from transforming the logical expression tree.
> >
> > Thanks in advance,
> > Milinda
> >
> > --
> > Milinda Pathirage
> >
> > PhD Student | Research Assistant
> > School of Informatics and Computing | Data to Insight Center
> > Indiana University
> >
> > twitter: milindalakmal
> > skype: milinda.pathirage
> > blog: http://milinda.pathirage.org
>
>


-- 
Milinda Pathirage

PhD Student | Research Assistant
School of Informatics and Computing | Data to Insight Center
Indiana University

twitter: milindalakmal
skype: milinda.pathirage
blog: http://milinda.pathirage.org

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message