drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jaimes, Rafael - 0993 - MITLL" <Rafael.Jai...@ll.mit.edu>
Subject Re: REST data source?
Date Tue, 31 Mar 2020 16:57:54 GMT
Hi Charles,

 

I am trying to use the http-storage plugin from your branch. I put the
storage plug-in files in a jar and tried to keep the jar directory structure
the same as other plug-ins. Upon starting drill-embedded I'm getting the
error below.  I am using your drill-module.conf and
bootstrap-storage-plugins.json from your branch. Is there another step I
need to perform to get Drill to recognize the plug-in? I am using 1.17
release.

 

Error: Failure in starting embedded Drillbit:
java.lang.IllegalStateException:
com.fasterxml.jackson.databind.exc.InvalidTypeIdException: Could not resolve
type id 'http' as a subtype of [simple type, class
org.apache.drill.common.logical.StoragePluginConfig]: known type ids =
[InfoSchemaConfig, SystemTablePluginConfig, file, hbase, hive, jdbc, kafka,
kudu, mock, mongo, named, openTSDB] (for POJO property 'storage')

at [Source: (String)"{

  "storage":{

    "http" : {

      "type":"http",

      "connections": {},

      "enabled": false

    }

  }

}

"; line: 4, column: 14] (through reference chain:
org.apache.drill.exec.planner.logical.StoragePlugins["storage"]->java.util.L
inkedHashMap["http"]) (state=,code=0)

 

Paul,

 

I don't know much about this REST service quite yet (it is internal).  We
utilize REST API where all responses are returned as JSON formatted strings
in many places, I don't think it is very sophisticated. I am not sure how it
will handle projection and filter issues. My current pipeline involves using
python requests.get() and then unpacking the response string. It does have
an authentication layer, so I am mildly concerned that the
HTTP-storage-plugin will have a hiccup - although it looks like it can use
"Basic". If I can get Drill to query the endpoint I will report back if I
find anything else that might be useful to you.

 

Thanks both for your great work with Drill!

 

-          Rafael


Mime
View raw message