drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Tucker <dtuc...@maprtech.com>
Subject Re: Drill connect to S3 with AWS EMR role
Date Tue, 12 May 2015 05:17:39 GMT
The current version of Hadoop in EMR (both Apache and MapR) does not support the IAM authentication
to S3 without the credentials in core-site.   I believe the support has been integrated into
Hadoop 2.6 … so when the EMR distributions upgrade to that level, the access you request
should be supported.

Did you successfully configure the drill-bit to use the full EMRFS jars, or did you default
to the older jets3t support ?   If you have the classpath settings for full emrfs support,
please share them with the group (and I will integrate that support into the MapR EMR bootstrap
action under development for Drill).


On May 11, 2015, at 6:55 PM, Alonzo Barnett <abarnett@lexmark.com> wrote:

> I am looking into using Drill with AWS EMR.  My organization uses IAM roles
> with EMR in order to rotate security credentials.
> In a first round of testing I successfully connected Drill to S3 on a
> cluster without IAM role based credential rotation which required placing
> access and secret keys in core-site.xml.
> It is possible to still use Drill with S3 without hardcoding credentials
> into core-site?  With emrfs there is some work by AWS to rotate
> credentials, and I would like to exploit the modification to core-site if
> possible.
> -- 
> v/r,
> Alonzo Barnett

View raw message