calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Muhammad Gelbana <m.gelb...@gmail.com>
Subject Re: Rewriting queries with Calcite
Date Mon, 06 May 2019 13:31:37 GMT
The method I've always followed so far to provide my rules, is to override
the AbstractRelNode.register method for the translated table node (Your
discovered tabels would implement the TranslatableTable interface and
implement the toRel method, check the Druid adapter for an example).

Thanks,
Gelbana


On Mon, May 6, 2019 at 12:33 PM Ivan Grgurina <Ivan.Grgurina@fer.hr> wrote:

> We found out that if SQL query is appropriately prepared (manually),
> Calcite can handle mapping it to multiple schemas. That is when using
> Calcite instead of JDBC directly in Java application.
>
> What ended up being the biggest mystery at the moment is how to, for
> example, integrate our custom ConverterRule so that application uses it
> when querying database. The idea being that we can force original query
> into the one where Calcite can handle mapping it to multiple databases.
>
> I'll give an example. Lets say you have 1 database at the start, with the
> appropriate Calcite schema. Lets say that database gets split up *vertically
> *by some external rules and you have their Calcite schemas.
> Then you create Java app that wants to target original 1 database and
> Calcite (some custom rules) convert it so it goes to multiple databases.
>
> So, if you have a query that goes something like this in original SQL
> format:
>
> "SELECT * FROM db.medinfo"
>
> , you would convert it into the query that goes something like this:
>
> "SELECT db1.medinfo1.id, db1.medinfo1.firstname, db1.medinfo1.lastname,
> db2.medinfo2.age, db3.medinfo3.illness
> FROM db1.medinfo1, db2.medinfo2, db3.medinfo3
> WHERE db1.medinfo1.id = db2.medinfo2.id AND db1.medinfo1.id =
> db3.medinfo3.id".
>
> My first question is how does Calcite recognize my custom rules? Is it by
> reflexion because I extended some class and then it automatically applies
> it, or is there some hook publish-subscribe system in place for extending
> Calcite with a library of custom rules?
> My second question is how can I make the above-mentioned application use
> my rules? Ergo, how do I connect Application-Library-Calcite?
>
> *Ivan Grgurina*
>
> Research Assistant (ZEMRIS)
> ------------------------------
>
> <https://www.linkedin.com/in/igrgurina/>
> <https://www.fer.unizg.hr/ivan.grgurina>
>
>
> ------------------------------
> *From:* Muhammad Gelbana <m.gelbana@gmail.com>
> *Sent:* Saturday, May 4, 2019 3:56 PM
> *To:* dev@calcite.apache.org
> *Subject:* Re: Rewriting queries with Calcite
>
> What do you mean by "rewrite queries to multiple data sources" ?
>
> Assuming you mean to run specific portions of the query plan against
> specific datasources, I beleive this can be done by changing the convention
> of the nodes for those protions. Each convention will map to a specific
> datasource. You can do that by writing converter rules.
>
> Assuming you mean to simply rewrite an SQL query, I beleive this can be
> done by visiting the root SQL node after parsing the query and rewrite the
> visitied nodes the way you wish. This actually can be done by other
> libraries than calcite, so I don't think that's what you're looking for.
>
> Assuming you mean to optimize the query plan, you'll need to write
> optimization rules (mentioned in the website's docs) to do that.
>
> As Stamatis said, a more specific question might get you a more specific
> answer.
>
> Thanks,
> Gelbana
>
>
> On Fri, May 3, 2019 at 3:22 PM Stamatis Zampetakis <zabetak@gmail.com>
> wrote:
>
> > Hi Ivan,
> >
> > It sounds like an interesting project, and I think Calcite will
> definitely
> > help you get there.
> >
> > However your questions are quite broad so it is difficult to provide a
> > concrete answer.
> > The best place to get started is the official website [1] where there
> are a
> > lot of examples and use-cases for Calcite.
> > Other than that there have been various discussions in the dev list such
> as
> > [2] where people have shared many useful resources.
> > Have a look and don't hesitate to come back to us.
> >
> > Good luck with your thesis!
> > Stamatis
> >
> > [1] https://calcite.apache.org/
> > [2]
> >
> >
> https://lists.apache.org/thread.html/3b32557adfc19e79e04a2d2e5ffcfa742c21e0fcfa3bd431025020ed@%3Cdev.calcite.apache.org%3E
> >
> > On Thu, May 2, 2019 at 11:30 AM Ivan Grgurina <Ivan.Grgurina@fer.hr>
> > wrote:
> >
> > > Hi, I'm working with Apache Calcite for my master thesis.
> > >
> > > The idea is to rewrite queries to multiple data sources, and Calcite
> is a
> > > serious candidate to be the tool for that job.
> > >
> > > At the moment, I'm trying to use Calcite as a library to create rules
> > that
> > > will rewrite queries during planner execution. I'm basing my current
> > > solution on https://github.com/tzolov/calcite-sql-rewriter code.
> > >
> > > My question would be if that's the best practice? What's the best way
> of
> > > creating a library based on Calcite that will be used in the end-user
> > > applications? What is the dev idea behind connecting that library to
> > > end-user application, as well as connecting it to Calcite?
> > >
> > > I have some idea of how that should work (based on above mentioned
> tool),
> > > but I would love to hear how do it properly from the devs.
> > >
> > > Thanks
> > >
> > >
> > > *Ivan Grgurina*
> > >
> > > Research Assistant (ZEMRIS)
> > > ------------------------------
> > >
> > > <https://www.linkedin.com/in/igrgurina/>
> > > <https://www.fer.unizg.hr/ivan.grgurina>
> > >
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message