drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kevin Lynx <kevinl...@gmail.com>
Subject Re: Implement a http service data source plugin
Date Wed, 27 May 2015 01:35:58 GMT
I tried

    select * from `/my/rest/endpoint` where $q = 'avi' and $p = 2 and count
> 0

to make all '$xxx' (starts with character '$') as query string, so i want
to convert the where clause to this:

  where count > 0

which means i want to consume "$q='avi' and $p=2". I write my own
FilterBuilder, and PushDownFilterForScan

    if (httpFilterBuilder.isAllExpressionsConverted()) {
      logger.debug("all expr converted");
      call.transformTo(newScanPrel);
    } else {
      logger.debug("not all expr converted");
      call.transformTo(filter.copy(filter.getTraitSet(),
ImmutableList.of((RelNode) newScanPrel)));
    }

But drill enters into a infinite loop.



------------------ Original ------------------
*From: * "Jacques Nadeau";<jacques@apache.org>;
*Date: * Tue, May 26, 2015 03:26 AM
*To: * "dev@drill.apache.org"<dev@drill.apache.org>;
*Subject: * Re: Implement a http service data source plugin

What about something like:

use http;
select * from `/my/rest/endpoint` where $q = 'avi' and $p = 2

Then in the storage plugin, we treat all $[name] values as virtual columns
that translate into query parameters?

On Mon, May 25, 2015 at 6:32 AM, kevin <kevinlynx@gmail.com> wrote:

> Hi,
>    I'm working on implementing a new data source plugin (StoragePlugin),
> to query data from a http service which response json data. At right now,
i
> can query like this:
>
>
>    select * from http.`q=avi&p=2` where count > 100
>
>
>    As you can see, i pass the http service query string by the table name
> `q=avi&p=2`, and which means the query string is STATIC. But in the real
> world a dynamic query string based on other query result is more useful.
> (Am i right ?)
>    If i want to support dynamic query string, does that mean i must
> implement StoragePluginOptimizerRule ? (which mongo storage plugin
> implemented, to transform query to mongodb query)
>    Say how about my query like this:
>
>
>    select * from http.dummy where reservsed_q='q=avi&p=2' and count > 100
>
>
>    I make a `reserved_q` to store the http query, and this string can be
> dynamic. And in my StoragePluginOptimizerRule, i must get `reserved_q`
> value, and consume the expression `reserved_q`.
>
>
>    So my question is, is this a good way (or suggest way) to do ? And can
> you guys give me some code hints to implement this, since
> StoragePluginOptimizerRule is a bit complicated to understand ?
>
>
> Thanks.

2015-05-26 3:26 GMT+08:00 Jacques Nadeau <jacques@apache.org>:

> What about something like:
>
> use http;
> select * from `/my/rest/endpoint` where $q = 'avi' and $p = 2
>
> Then in the storage plugin, we treat all $[name] values as virtual columns
> that translate into query parameters?
>
> On Mon, May 25, 2015 at 6:32 AM, kevin <kevinlynx@gmail.com> wrote:
>
> > Hi,
> >    I'm working on implementing a new data source plugin (StoragePlugin),
> > to query data from a http service which response json data. At right
> now, i
> > can query like this:
> >
> >
> >    select * from http.`q=avi&p=2` where count > 100
> >
> >
> >    As you can see, i pass the http service query string by the table name
> > `q=avi&p=2`, and which means the query string is STATIC. But in the real
> > world a dynamic query string based on other query result is more useful.
> > (Am i right ?)
> >    If i want to support dynamic query string, does that mean i must
> > implement StoragePluginOptimizerRule ? (which mongo storage plugin
> > implemented, to transform query to mongodb query)
> >    Say how about my query like this:
> >
> >
> >    select * from http.dummy where reservsed_q='q=avi&p=2' and count > 100
> >
> >
> >    I make a `reserved_q` to store the http query, and this string can be
> > dynamic. And in my StoragePluginOptimizerRule, i must get `reserved_q`
> > value, and consume the expression `reserved_q`.
> >
> >
> >    So my question is, is this a good way (or suggest way) to do ? And can
> > you guys give me some code hints to implement this, since
> > StoragePluginOptimizerRule is a bit complicated to understand ?
> >
> >
> > Thanks.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message