calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Julian Hyde <jh...@apache.org>
Subject Re: Adjust clacite to support kylin query
Date Mon, 05 Feb 2018 19:53:48 GMT
If the query requires more than one data source (Kylin and something else) then a logical plan
is necessary. If it’s just Kylin then what you suggest would work in principle, but it would
be hard to make it work in practice.

It might be simpler to fix the SQL generated by Calcite’s JDBC adapter to be what Kylin
expects. Have you tried adding a Kylin dialect, making sure it gets used by RelToSqlConverter
/ JdbcImplementor?

Julian


> On Feb 4, 2018, at 9:53 PM, Neters <neters@foxmail.com> wrote:
> 
> Dear Julian,
> 
> 
> Have a good day.
> 
> 
> Yes. I have tried to connect to kylin via JDBC adapter, but the sql string has been changed
to which kylin engine does not support(sql string was parsed not correctly ). 
> 
> 
> The sql query process i understand is that : 
> 
> 
> avatica jdbc driver gets connection and sumit sql string -->avatica server receives
the sql string->calcite convertes the sql string to a logical plan-->
> calcite applys planning rules to optimize the query process--> calcite converts the
sql string back and submits it to the database.
> 
> 
> The description above is right?
> 
> 
> I am wondering if it is possible to bypass the logical plan convertion. After avatica
server receives the sql string, calcite posts the sql string to kylin rest server by kylin
jdbc driver defined in the json model file? As you can see, the implements of kylin jdbc is
as following:
> 1.register kylin jdbc driver
> 2.get statement
> 3.set query sql string and execute query( excute query by posting sql string to kylin
rest server)
> 
> 
> If bypass issue could be carried out to query kylin, there might be a point to bypass
the convertion to logical plan.
> 
> 
> 
> 
> ----------------------------------------------------------------------------------------------------------------------------
> 
> 
> My avatica server source archive is as following:
> ----project name----
>                     --java code--
>                            +BiQueryServer.java
>                            +ModelMetaFactory.java
>                    --resources--
>                            +model.json
> 
> 
> --------------------------------------------BiQueryServer.java---------------------------------------------------------------------
> public class BiQueryServer {
>    public static void main(String[] args) {
>        List<String> argsList = new ArrayList<String>();
>        argsList.add("com.w3sun.sqlengine.ModelMetaFactory");
>        argsList.add("org.apache.calcite.avatica.jdbc.JdbcMeta");
> 
> 
>        for (String arg : args) {
>            argsList.add(arg);
>        }
> 
> 
>        String[] argsArray = (String[]) argsList.toArray(new String[argsList.size()]);
>        try {
>            org.apache.calcite.avatica.server.Main.main(argsArray);
>        } catch (InterruptedException e) {
>            e.printStackTrace();
>        } catch (ClassNotFoundException e) {
>            e.printStackTrace();
>        } catch (IllegalAccessException e) {
>            e.printStackTrace();
>        } catch (InstantiationException e) {
>            e.printStackTrace();
>        }
>    }
> }
> 
> ------------------------------------------------ModelMetaFactory.java-----------------------------------------------------------------------------
> public class ModelMetaFactory implements Meta.Factory {
>    private final String modelFile = this.getClass().getResource("/model.json").getPath().substring(1);
>    private final String localJdbcUrl = "jdbc:calcite:";
> 
> 
>    public void printMetaInfo(String meta) {
>        System.out.println("[TnpmConnectionFactory.create] Implement with " + meta);
> 
> 
>    }
> 
> 
>    public Meta create(List<String> list) {
>        try {
>            if (list.size() > 0 && list.get(0).equals("org.apache.calcite.avatica.jdbc.JdbcMeta"))
{
>                return new JdbcMeta(localJdbcUrl + "model=" + (list.size() > 1 ? list.get(1)
: modelFile));
>            } else {
>                return null;
>            }
>        } catch (SQLException e) {
>            throw new RuntimeException(e);
>        }
>    }
> }
> 
> ----------------------------------------------model.json------------------------------------------------------
> {
>  "version": "1.0",
>  "defaultSchema": "kylin",
>  "schemas": [
>    {
>      "name": "kylin",
>      "type": "custom",
>      "factory": "org.apache.calcite.adapter.jdbc.JdbcSchema$Factory",
>      "operand": {
>        "jdbcDriver": "org.apache.kylin.jdbc.Driver",
>        "jdbcUrl":"jdbc:kylin://192.18.23.15:7070/test_project",
>        "jdbcUser": "ADMIN",
>        "jdbcPassword": "KYLIN"
>      }
>    }
>  ]
> }
> 
> 
> 
> 
> 
> Best Regards
> 
> 
> Neters
> 
> 
> ------------------ 原始邮件 ------------------
> 发件人: "Julian Hyde";<jhyde@apache.org>;
> 发送时间: 2018年2月3日(星期六) 凌晨2:31
> 收件人: "dev"<dev@calcite.apache.org>;
> 
> 主题: Re: Adjust clacite to support kylin query
> 
> 
> 
> Have you tried connecting to Kylin (and MySQL) via the JDBC adapter?
> 
> (I know Kylin has Calcite inside, but they don’t package it as an adapter, so we’re
forced to treat them as a black box.)
> 
> Julian
> 
> 
>> On Feb 2, 2018, at 4:13 AM, Neters <neters@foxmail.com> wrote:
>> 
>> Hi guys,
>> 
>> Good day.
>> 
>> Sorry guys,the mail system remove my pic, so i make it attachment.
>> 
>> I want to query apache kylin by calcite,but i did not find the adapter for kylin.
So i decide to modify the code attached in kylin source code.
>> My frame like this:
>> 
>>                              avatica jdbc client      
>>                                         |
>>                                         |
>>                            -------------------------
>>                           |          calcite           |
>>                           --------------------------
>>                           /               |               \
>>                     kylin              es               mysql
>> 
>> 
>> 
>> 
>> Could you please give some constructive support?
>> 
>> Best Regards
>> 


Mime
View raw message