drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andries Engelbrecht <aengelbre...@mapr.com>
Subject Re: Issue Drill querying S3 recursive
Date Thu, 12 Jan 2017 15:44:48 GMT
You may want to upgrade to the latest Drill version (1.9).

Also what is the storage plugin called dfs, fs,...?

If it is called dfs then you can thry the following.

use dfs;

show files;

This will show you the files and directories in the plugin root, you can navigate from there.

If it shows you t as a directory you can then see which files are in t;

show files in `/t`;

See if the csv file shows up, if so this should work.

select * from dfs.`/t`;


From: Magesh Waran <mageshwar.m@gmail.com>
Sent: Thursday, January 12, 2017 6:31:22 AM
To: user@drill.apache.org
Subject: Issue Drill querying S3 recursive

Hi Team,

I am trying to query a file under folder 't/atms-csv.csv' which I can successfully do it.

[Inline image 1]

There is another file in that location which as additional data from another day (both file
columnmodel). when I try query recursively using folder name 't' which shows below error
[Inline image 2]
Tried queries (All doesnt worked out):
1. select * from `t/`;
2. select * from `/t/`;
3. select * from dfs.`t/`;
4. select * from dfs`/t/`;
5. select * from fs.`t/`;
6. select * from fs`/t/`;

I am using below versions,
Drill - 1.6.0
S3 protool: S3n (also tried S3, S3a)
Sample Query String:
 "connection": "s3a://xxxxx-bucket",
  "config": {
    "fs.s3a.access.key": "xxxxxxxxxxxxxxxxxxxxxxxxxxxx",
    "fs.s3a.secret.key": "xxxxxxxxxxxxxxxxxxxxxxxxxxxx"
  "workspaces": {
    "root": {
      "location": "/",
      "writable": true,
      "defaultInputFormat": null

Thanks & Regards
Mageshwaran M
Contact: +91-9944667832

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message