drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andries Engelbrecht <aengelbre...@mapr.com>
Subject Re: Connecting to S3 bucket which does not seem to require a key
Date Mon, 12 Jun 2017 14:43:14 GMT
You may be better of downloading the NYC bike data set locally and convert to parquet.
Converting from csv.zip to parquet will result in large improvements in performance if you
do various queries on the data set.


On 6/11/17, 10:48 PM, "Abhishek Girish" <agirish@apache.org> wrote:

    Drill connects to to S3 buckets (AWS) via the S3a library. And the storage
    plugin configuration requires the access & secret keys [1].
    I'm not sure if Drill can access S3 without the credentials. It might be
    possible via custom authenticators [2]. Hopefully others who have tried
    this will comment.
    [1] https://drill.apache.org/docs/s3-storage-plugin/
    [2] http://docs.aws.amazon.com/AmazonS3/latest/API/sig-
    On Wed, Jun 7, 2017 at 3:02 PM, Jack Ingoldsby <jack.ingoldsby@gmail.com>
    > Hi,
    > I'm trying to access the NYC Citibike S3 bucket, which seems to publicly
    > available
    > https://s3.amazonaws.com/tripdata/index.html
    > If I leave the Access Key & Secret Key empty, I get the following message
    > 0: jdbc:drill:zk=local> !tables
    > Error: Failure getting metadata: Unable to load AWS credentials from any
    > provider in the chain (state=,code=0)
    > If I try entering random numbers as keys, I get the following message
    > Error: Failure getting metadata: Status Code: 403, AWS Service: Amazon S3,
    > AWS Request ID: 1C888A3A21D79F87, AWS Error Code: InvalidAccessKeyId, AWS
    > Error Message: The AWS Access Key Id you provided does not exist in our
    > records. (state=,code=0)
    > Is it possible to connect to a data source that does not seem to require a
    > key?
    > Thanks,
    > Jack

View raw message