drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andy Grove <andygrov...@gmail.com>
Subject Re: Looking for advice on integrating with a custom data source
Date Sun, 12 Jan 2020 21:03:19 GMT
Between reading the tutorial and copying and pasting code from the Kudu
storage plugin, I've been making reasonable progress with this but am I but
confused by one error I'm now hitting.
ExecutionSetupException: Failure finding OperatorCreator constructor for
config com.mydb.MyDbSubScan
Prior to this, Drill had called getSpecificScan and then called a few of
the methods on my subscan object. I wasn't sure what to return for
getOperatorType so just returned the kudu subscan operator type and I'm
wondering if the issue is related to that somehow?

Thanks.


On Sat, Jan 11, 2020 at 10:13 PM Andy Grove <andygrove73@gmail.com> wrote:

> Thank you both for the those responses. This is very helpful. I have
> ordered a copy of the book too. I'm using Drill 1.17.0.
>
> I'll take a look at the Jdbc Storage Plugin code and see if it would be
> feasible to add the logic I need there. In parallel, I've started
> implementing a new storage plugin. I'll be working on this more tomorrow
> and I'm sure I'll be back with more questions soon.
>
> Thanks again for your help!
>
> Andy.
>
>
>
>
>
>
>
> On Sat, Jan 11, 2020 at 6:03 PM Charles Givre <cgivre@gmail.com> wrote:
>
>> HI Andy,
>> Thanks for your interest in Drill.  I'm glad to see that Paul wrote you
>> back as well.  I was going to say I thought the JDBC storage plugin did in
>> fact push down columns and filters to the source system.
>>
>> Also, what version of Drill are you using?
>>
>> Writing a storage plugin for Drill is not trivial and I'd definitely
>> recommend using the code from Paul's PR as that greatly simplifies things.
>> Here is a tutorial as well:
>> https://github.com/paul-rogers/drill/wiki/Create-a-Storage-Plugin
>>
>> If you need additional help, please let us know.
>> -- C
>>
>>
>> On Jan 11, 2020, at 5:57 PM, Andy Grove <andygrove73@gmail.com> wrote:
>>
>> Hi,
>>
>> I'd like to use Apache Drill with a custom data source that supports a
>> subset of SQL.
>>
>> My goal is to have Drill push selection and predicates down to my data
>> source but the rest of the query processing should take place in Drill.
>>
>> I started out by writing a JDBC driver for the data source and registering
>> that with Drill using the Jdbc Storage Plugin but it seems to just pass
>> the
>> whole query through to my data source, so that approach isn't going to
>> work
>> unless I'm missing something?
>>
>> Is there any way to configure the JDBC storage plugin to only push certain
>> parts of the query to the data source?
>>
>> If this isn't a good approach, do I need to write a custom storage plugin?
>> Can these be added on the classpath or would that require me maintaining a
>> fork of the project?
>>
>>
>>
>> I appreciate any pointers anyone can give me.
>>
>> Thanks,
>>
>> Andy.
>>
>>
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message