drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alonzo Barnett <abarn...@lexmark.com>
Subject RE: Drill connect to S3 with AWS EMR role
Date Tue, 12 May 2015 13:13:10 GMT
 Thank you for the help.

I am using the jetS3t support. I will look into setting up emrfs support
today, though, and share anything I come up with.

Sent via the Samsung Galaxy Note® 4, an AT&T 4G LTE smartphone

-------- Original message --------
From: David Tucker <dtucker@maprtech.com>
Date: 05/12/2015 1:18 AM (GMT-05:00)
To: user@drill.apache.org
Subject: Re: Drill connect to S3 with AWS EMR role

The current version of Hadoop in EMR (both Apache and MapR) does not
support the IAM authentication to S3 without the credentials in core-site.
 I believe the support has been integrated into Hadoop 2.6 … so when the
EMR distributions upgrade to that level, the access you request should be

Did you successfully configure the drill-bit to use the full EMRFS jars, or
did you default to the older jets3t support ?   If you have the classpath
settings for full emrfs support, please share them with the group (and I
will integrate that support into the MapR EMR bootstrap action under
development for Drill).


On May 11, 2015, at 6:55 PM, Alonzo Barnett <abarnett@lexmark.com> wrote:

> I am looking into using Drill with AWS EMR.  My organization uses IAM
> with EMR in order to rotate security credentials.
> In a first round of testing I successfully connected Drill to S3 on a
> cluster without IAM role based credential rotation which required placing
> access and secret keys in core-site.xml.
> It is possible to still use Drill with S3 without hardcoding credentials
> into core-site?  With emrfs there is some work by AWS to rotate
> credentials, and I would like to exploit the modification to core-site if
> possible.
> --
> v/r,
> Alonzo Barnett

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message