hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [hadoop] steveloughran opened a new pull request #879: HADOOP-15563 Full S3Guard Support for on-demand DDB tables
Date Fri, 31 May 2019 15:45:13 GMT
steveloughran opened a new pull request #879: HADOOP-15563 Full S3Guard Support for on-demand
DDB tables
URL: https://github.com/apache/hadoop/pull/879
 
 
   HADOOP-15563 Full S3Guard Support for on-demand DDB tables
   
   * If you create a table with read capacity == write capacity == 0, you get an on-demand
table.
   * change the defaults in Constants and core-defaults to be zero. You get an on-demand table
by default.
   * S3Guard docs reworked to cover the topic, including the updated defaults and why you
should go to on-demand; also mentioned in the troubleshooting section.
   
   It'd be nice if set-capacity could switch a table to on-demand, but the latest AWS SDK
doesn't let you do that, even though the low level REST API does.
   
   ITestS3GuardToolDynamoDB l
   * ifecycle test creates an on-demand table and skips changing its capacity.
   * Fix HADOOP-16187 ITestS3GuardToolDynamoDB test failures: test wasn't clearing enough
per-bucket options.
   
   ITestDynamoDBMetadataStore: review of every test case in so that
   * Capacity is always set to on-demand before table creation
   * Every test always cleans up its tables, even on assert failures.
   
   Those tests have tended to leak tables in the past; these changes should eliminate it on
uninterrupted test runs, and with on-demand capacity you don't get billed much until the next
time you do an audit and delete of tables.
   
   Also:
   
   - removed @Unstable attribute from those Constants which I consider having been shipping
too long to be changed.
   - moved test-only S3GUARD_DDB_TEST_TABLE_NAME_KEY key to S3ATestConstants. It was tagged
Unstable, and should be in the right file.
   
   Change-Id: Ia44559354666db8027e574eab97983167ed930bf
   
   Note: this patch includes HADOOP-16117 as precursor; that's a separate PR, (#818); its
needed to be able to create on-demand tables

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message