drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shankar Mane <shankar.m...@games24x7.com>
Subject Re: QUESTION: Drill Configuration to access S3 buckets
Date Thu, 15 Jun 2017 04:36:54 GMT
aws new regions uses only signature version 4 protocol for S3. Other
regions has both V2 and V4 compatible. Drill works very well if regions has
both signature versions.

By adding endpoints, same problem persists. May be Drill API doesn't have
support to V4 protocol yet.

This V4 problems is also with native hadoop versions prior to 2.8.0.



On 15-Jun-2017 9:49 AM, "Jack Ingoldsby" <jack.ingoldsby@gmail.com> wrote:

> Useful to know, thanks. Also having problems with Ohio. Will try another
> region
>
> On Wed, Jun 14, 2017, 19:46 Сергей Боровик <borovyksv@gmail.com> wrote:
>
> > Hi!
> > I have an AWS EC2 instance with apache drill 1-10.0.and configured IAM
> > Role.
> >
> > And I am able to access and query S3 bucket in US East (N. Virginia)
> > region,
> > but not able to access/query buckets in US East (Ohio) region, it fails
> > with
> > "error: system error: amazons3exception: status code 400, AWS Service:
> > Amazon S3,
> > AWS Request ID:9D54A8310F26582B, AWS Error Code: null, AWS Error Message:
> > Bad Request"
> >
> >
> > I've tried set conf/core-site.xml property to:
> >
> > <property>
> >     <name>fs.s3a.endpoint</name>
> >     <value>s3.us-east-2.amazonaws.com</value>
> > </property>
> >
> > in this case Ohio fails with the same error,
> > and N. Virginia has error status code 301, AWS Error Code:
> > PermanentRedirect,
> > AWS Error message: The bucket you are attempting to access must be
> > addressed using the specified endpoint
> >
> > 1) Is there any specific configuration that needs to be enabled on Drill
> > for Ohio region?
> > 2) Does Drill not work on aws signature version 4?
> >
> > Thank you in advance.
> > Any advice is much appreciated!
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message