hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-16477) S3 delegation token tests fail if fs.s3a.encryption.key set
Date Thu, 01 Aug 2019 13:14:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-16477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16898075#comment-16898075
] 

Steve Loughran commented on HADOOP-16477:
-----------------------------------------

unsetting things works for the basic test setup here.

Even there though, the test {{ITestRoleDelegationInFileystem.testDelegatedFileSystem}} fails
when writing an object, 403.

Assumption: the role policy created for the DT doesn't include the permission to encrypt data
with SSE-KMS. Which would count as a bug in the code, rather than just test setup

{code}
2019-08-01 14:08:37,777 [teardown] INFO  contract.AbstractFSContractTestBase (AbstractFSContractTestBase.java:describe(255))
- closing file system

java.nio.file.AccessDeniedException: test/testDTFileSystemClient: put on test/testDTFileSystemClient:
com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon S3; Status
Code: 403; Error Code: AccessDenied; Request ID: E86544FF1D029857; S3 Extended Request ID:
TDRYwBniy+TkRNAx3Mqn57CY/m5ggxuJA6FgizI6koEwWJkqKo7vHst6tQZAaHX7gXp/jlDREaw=), S3 Extended
Request ID: TDRYwBniy+TkRNAx3Mqn57CY/m5ggxuJA6FgizI6koEwWJkqKo7vHst6tQZAaHX7gXp/jlDREaw=:AccessDenied

	at org.apache.hadoop.fs.s3a.S3AUtils.translateException(S3AUtils.java:243)
	at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:111)
	at org.apache.hadoop.fs.s3a.Invoker.lambda$retry$4(Invoker.java:314)
	at org.apache.hadoop.fs.s3a.Invoker.retryUntranslated(Invoker.java:406)
	at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:310)
	at org.apache.hadoop.fs.s3a.Invoker.retry(Invoker.java:285)
	at org.apache.hadoop.fs.s3a.WriteOperationHelper.retry(WriteOperationHelper.java:150)
	at org.apache.hadoop.fs.s3a.WriteOperationHelper.putObject(WriteOperationHelper.java:460)
	at org.apache.hadoop.fs.s3a.S3ABlockOutputStream.lambda$putObject$0(S3ABlockOutputStream.java:438)
	at org.apache.hadoop.util.SemaphoredDelegatingExecutor$CallableWithPermitRelease.call(SemaphoredDelegatingExecutor.java:219)
	at org.apache.hadoop.util.SemaphoredDelegatingExecutor$CallableWithPermitRelease.call(SemaphoredDelegatingExecutor.java:219)
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: com.amazonaws.services.s3.model.AmazonS3Exception: Access Denied (Service: Amazon
S3; Status Code: 403; Error Code: AccessDenied; Request ID: E86544FF1D029857; S3 Extended
Request ID: TDRYwBniy+TkRNAx3Mqn57CY/m5ggxuJA6FgizI6koEwWJkqKo7vHst6tQZAaHX7gXp/jlDREaw=),
S3 Extended Request ID: TDRYwBniy+TkRNAx3Mqn57CY/m5ggxuJA6FgizI6koEwWJkqKo7vHst6tQZAaHX7gXp/jlDREaw=
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1712)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1367)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1113)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:686)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4920)
	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4866)
	at com.amazonaws.services.s3.AmazonS3Client.access$300(AmazonS3Client.java:389)
	at com.amazonaws.services.s3.AmazonS3Client$PutObjectStrategy.invokeServiceCall(AmazonS3Client.java:5800)
	at com.amazonaws.services.s3.AmazonS3Client.uploadObject(AmazonS3Client.java:1789)
	at com.amazonaws.services.s3.AmazonS3Client.putObject(AmazonS3Client.java:1749)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.putObjectDirect(S3AFileSystem.java:1935)
	at org.apache.hadoop.fs.s3a.WriteOperationHelper.lambda$putObject$5(WriteOperationHelper.java:462)
	at org.apache.hadoop.fs.s3a.Invoker.once(Invoker.java:109)
	... 15 more
{code}

> S3 delegation token tests fail if fs.s3a.encryption.key set
> -----------------------------------------------------------
>
>                 Key: HADOOP-16477
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16477
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3, test
>    Affects Versions: 3.3.0
>            Reporter: Steve Loughran
>            Priority: Major
>
> if you set an s3a encryption key, the Session and Role DelegationToken tests fail...the
test setup needs to unset that key for config and bucket



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message