hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15837) DynamoDB table Update can fail S3A FS init
Date Tue, 09 Oct 2018 22:13:03 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644159#comment-16644159
] 

Steve Loughran commented on HADOOP-15837:
-----------------------------------------

Patch 001. 
# change case statement so that updating == active as far as init is concerned, consistent
with the AWS docs
# error handing/retries around waitForTable access, with tests

I'd actually implemented change #2 and was testing it before I found the AWS docs which said
yes, updating is good to go —the most minimal patch is just that switch statement change.
I'm putting this patch here fore review and will then roll it back. Table creation is rare
enough in production that retries there aren't a real need; the use case "on-demand-create"
is more a test/explorative option.

Tested. Yes, but only as part of HADOOP-14556, and not (yet) successfully recreated the updating
failure (or it did happen but now it didn't fail)

> DynamoDB table Update can fail S3A FS init
> ------------------------------------------
>
>                 Key: HADOOP-15837
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15837
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.2.0
>         Environment: s3guard test with small capacity (10) but autoscale enabled &
multiple consecutive parallel test runs executed...this seems to have been enough load to
trigger the state change
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>            Priority: Major
>         Attachments: HADOOP-15837-001.patch
>
>
> When DDB autoscales a table, it goes into an UPDATING state. The waitForTableActive operation
in the AWS SDK doesn't seem to wait long enough for this to recover. We need to catch &
retry



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message