drill-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andries Engelbrecht <aengelbre...@maprtech.com>
Subject Re: S3 bucket configuration security error on the storage plugin
Date Thu, 04 Aug 2016 16:05:23 GMT
Also if you want to put the key in the plugin config the syntax is as follows

  "config": {
    "fs.s3a.access.key": "<PUT YOUR ACCESS KEY HERE>",
    "fs.s3a.secret.key": "<PUT YOUR SECRET KEY HERE>"
  },


--Andries


> On Aug 4, 2016, at 7:55 AM, Andries Engelbrecht <aengelbrecht@maprtech.com> wrote:
> 
> You have to put the actual ID and KEY in the storage plugin (not recommended per se if
you don't have the WebUI secured).
> 
> Or you can edit the core-site.xml file in the drill conf directory to add the s3a ID
and KEY there.
> 
> Also use s3a not s3 in the later versions of Drill.
> 
> http://drill.apache.org/docs/s3-storage-plugin/ <http://drill.apache.org/docs/s3-storage-plugin/>
> 
> --Andries
> 
> 
> 
> 
>> On Aug 3, 2016, at 5:48 PM, <mdhruv1@yahoo.com.INVALID <mailto:mdhruv1@yahoo.com.invalid>>
<mdhruv1@yahoo.com.INVALID <mailto:mdhruv1@yahoo.com.invalid>> wrote:
>> 
>> 
>> Hi,
>> I am trying to configure an s3 bucket and I am getting this error. Can any one help
me
>> org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: IllegalArgumentException:
AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively)
of a s3 URL, or by setting the fs.s3.awsAccessKeyId or fs.s3.awsSecretAccessKey properties
(respectively).[Error Id: 02956e73-87a8-4279-b6db-cb89d11821b8 on 10.64.86.175:31010] 
>> 
>> The bucket name is itgs-hiss-reporting
>>  are the sub folder in it.
>> /staging/oneems
>> The storage plugin  is{
>>   "type": "file",
>>   "enabled": true,
>>   "connection": "s3://itgs-hiss-reporting <s3://itgs-hiss-reporting>",
>>   "config": {
>>     "fs.s3a.awsAccessKeyId": "id",
>>     "fs.s3a.awsSecretAccessKey": "key"
>>   },
>>   "workspaces": {
>>     "root": {
>>       "location": "/staging",
>>       "writable": false,
>>       "defaultInputFormat": "csv"
>>     },
>> the core-site.xml config is
>> <configuration>
>> 
>>     <property>
>>         <name>fs.s3a.access.key</name>
>>         <value>ID</value>
>>     </property>
>> 
>>     <property>
>>         <name>fs.s3a.secret.key</name>
>>         <value>Secret Key</value>
>>                
>>     </property>
>> 
>> </configuration>
>> 
>> When I got to drill ip:8047 
>> 
>> and then do a show databases I get this error
>> org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: IllegalArgumentException:
AWS Access Key ID and Secret Access Key must be specified as the username or password (respectively)
of a s3 URL, or by setting the fs.s3.awsAccessKeyId or fs.s3.awsSecretAccessKey properties
(respectively).[Error Id: 352382c5-76ec-4253-b323-35e167f14db6 
>> 
>> 
>> 
>> 
>> 
> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message