calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Julian Hyde <jh...@apache.org>
Subject Re: Google Summer of Code 2018
Date Wed, 07 Mar 2018 21:47:32 GMT
That would be excellent. Some of the aspects of the paper will be interesting to implement
in Calcite - e.g. piecewise linear cost functions.

Can we use existing RelOptRule instances unchanged in MOP (multi-objective parametric) mode?
It remains to be seen, but I hope so, because our RelOptRules are a huge asset.

I would also be interested in optimizing queries with 0 parameters (i.e. what Calcite does
today), keep the planning state, then add parameters.

Calcite’s cost model is currently broken. RelOptCost is a vector (cpu, io, rows) and therefore
is a partial order, whereas Volcano requires a total order. If we are able to convert cost
into a scalar, c = p0 * cpu + p1 * io + p2 * rows, where p0, p1 and p2 are parameters, then
we now have a total order. With MOP, we would still want to do multi-phase optimization, and
we could set some parameters to zero in particular phases to reduce the size of the search
space.

Lastly, I would like to tie in Jesus's work on intermediate results where he used a {0, 1}
parameter to represent whether a particular intermediate result was populated. He used an
ILP solver to find the optimal set of intermediate results to populate but if we were to make
Jesus’s parameters into parameters of the MOP model, my hunch is that MOP would be able
to find optimal values, i.e. figure out which intermediate results to materialize.


> On Mar 7, 2018, at 6:28 AM, Michael Mior <mmior@uwaterloo.ca> wrote:
> 
> Just thought I would point out that I have a fellow student here who seems
> interested in trying to tackle implementing on multi-objective parametric
> query optimization in Calcite. If anyone else would like to be involved in
> that effort, let me know. Although this wouldn't be a GSoC project, I
> believe he's willing to contribute a significant amount of time over the
> summer.
> 
> --
> Michael Mior
> mmior@apache.org
> 
> 2018-02-27 10:24 GMT-05:00 Edmon Begoli <ebegoli@gmail.com>:
> 
>> If we are to be ambitious, and to give someone a real challenge --
>> implement this approach for Calcite:
>> Multi-objective Parametric Query Optimization
>> https://cacm.acm.org/magazines/2017/10/221322-multi-objective-parametric-
>> query-optimization/fulltext
>> 
>> (I think Julian might have mentioned this once before in a different
>> context. I might be wrong ...)
>> 
>> On Tue, Feb 27, 2018 at 10:20 AM, Michael Mior <mmior@uwaterloo.ca> wrote:
>> 
>>> I would suggest creating a JIRA case to track this. It sounds like a
>> basic
>>> implementation could be of reasonable scope for a GSoC project.
>>> 
>>> --
>>> Michael Mior
>>> mmior@apache.org
>>> 
>>> 2018-02-26 19:09 GMT-05:00 Chunhui Shi <cshi@mapr.com>:
>>> 
>>>> What about "SQL support for JavaScript Object Notation (JSON)"?
>>>> www.iso.org/standard/67367.html<http://www.iso.org/standard/67367.html
>>> ?
>>>> 
>>>> There are two categories of JSON functions (8 functions) for query and
>>> for
>>>> constructor, some clauses like PLAN, etc are introduced in this
>> standard.
>>>> 
>>>> I did not see the supports in mainstream DBs(Oracle and Sql Server)
>>>> covering all of them, but IS JSON, JSON_QUERY and JSON_VALUE are both
>>>> implemented
>>>> 
>>>> SQL server support
>>>> https://docs.microsoft.com/en-us/sql/relational-databases/
>>>> json/validate-query-and-change-json-data-with-built-
>>>> in-functions-sql-server
>>>> 
>>>> Oracle:
>>>> https://docs.oracle.com/database/121/ADXDB/json.htm#ADXDB6374
>>>> 
>>>> MySQL and Postgres have a lot functions to handle JSON,  besides
>>>> JSON_OBJECT or one or two other functions, seems none of its function
>> is
>>>> standardized.
>>>> https://dev.mysql.com/doc/refman/5.7/en/json-function-reference.html
>>>> 
>>>> Postgres
>>>> <https://dev.mysql.com/doc/refman/5.7/en/json-function-reference.html>
>>>> https://www.postgresql.org/docs/current/static/functions-json.html
>>>> 
>>>> 
>>>> While I think the scope could be self contained, if it is not suitable
>>> for
>>>> this activity do we want to put this into Calcite roadmap?
>>>> 
>>>> 
>>>> Best,
>>>> 
>>>> Chunhui
>>>> 
>>>> ________________________________
>>>> From: Julian Hyde <jhyde@apache.org>
>>>> Sent: Monday, February 26, 2018 4:00:21 PM
>>>> To: dev
>>>> Subject: Re: Google Summer of Code 2018
>>>> 
>>>> Yes.
>>>> 
>>>> 
>>>>> On Feb 26, 2018, at 2:32 PM, Michael Mior <mmior@uwaterloo.ca>
>> wrote:
>>>>> 
>>>>> Thanks Julian! Would these be some good JIRAs to tag?
>>>>> 
>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
>>>> apache.org_jira_browse_CALCITE-2D1737&d=DwICAg&c=
>>> cskdkSMqhcnjZxdQVpwTXg&r=
>>>> 7bZjGOKpQi7qeyQ_xRpzEQ&m=LYUbb9SvNkKb5PpYzt7-
>> eWRg5qGj9K6txN2zGGcqakg&s=
>>>> KXsK-qlYTOf5W1KDQmlqjA7CIj4_CtGlmWirBlPxopI&e=
>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
>>>> apache.org_jira_browse_CALCITE-2D1861&d=DwICAg&c=
>>> cskdkSMqhcnjZxdQVpwTXg&r=
>>>> 7bZjGOKpQi7qeyQ_xRpzEQ&m=LYUbb9SvNkKb5PpYzt7-
>> eWRg5qGj9K6txN2zGGcqakg&s=
>>>> 6xfRRfhSus-SrVHKvEIYKzcyAaEAUJBLdHbkqjUhrRk&e=
>>>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__issues.
>>>> apache.org_jira_browse_CALCITE-2D2031&d=DwICAg&c=
>>> cskdkSMqhcnjZxdQVpwTXg&r=
>>>> 7bZjGOKpQi7qeyQ_xRpzEQ&m=LYUbb9SvNkKb5PpYzt7-
>>>> eWRg5qGj9K6txN2zGGcqakg&s=oLu_OiKf-cFe1RxcmAnv2CFW6QPGbpz5uYeiN5y
>> 3EMs&e=
>>>>> 
>>>>> --
>>>>> Michael Mior
>>>>> mmior@apache.org
>>>>> 
>>>>> 2018-02-26 14:46 GMT-05:00 Julian Hyde <jhyde@apache.org>:
>>>>> 
>>>>>> Here are two areas that are self-contained and rewarding:
>>>>>> * Spatial functions
>>>>>> * Spark adapter
>>>>>> 
>>>>>> Julian
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> On Feb 24, 2018, at 4:24 PM, Michael Mior <mmior@uwaterloo.ca>
>>> wrote:
>>>>>>> 
>>>>>>> You have probably seen that Apache was accepted as an organization
>>> for
>>>>>> this
>>>>>>> year's GSoC. I thought I would see if anyone in the Calcite
>> community
>>>> can
>>>>>>> think of any issues that would be a good fit. It's no guarantee
we
>>>> would
>>>>>>> get someone to work on it, but it could be a good push to move
some
>>>>>>> isolated bits of functionality forward that may not get much
>>> attention
>>>>>>> otherwise.
>>>>>>> 
>>>>>>> --
>>>>>>> Michael Mior
>>>>>>> mmior@apache.org
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>>> 
>> 


Mime
View raw message