hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-13171) Add StorageStatistics to S3A; instrument some more operations
Date Mon, 23 May 2016 18:43:13 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-13171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Steve Loughran updated HADOOP-13171:
------------------------------------
    Attachment: HADOOP-13171-branch-2-005.patch

 Patch 005: statistics on put and delete operations, more overloaded methods (exists,() isDirectory()),
test on copyFileToLocal to verify that the method works with the metrics updated in the process.

the ProgressListener class has been factored out and cleaned up for easier reporting; in the
process the various output streams have had a lot of their use of the s3a client moved into
the S3AFilesystem; public methods are provided which include the instrumentation counter updates.

Also fixes a possible race condition wherein the creation of empty directories was not being
awaited on, which may have been a trigger for some intermittent race conditions

> Add StorageStatistics to S3A; instrument some more operations
> -------------------------------------------------------------
>
>                 Key: HADOOP-13171
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13171
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 2.8.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>            Priority: Minor
>         Attachments: HADOOP-13171-branch-2-001.patch, HADOOP-13171-branch-2-002.patch,
HADOOP-13171-branch-2-003.patch, HADOOP-13171-branch-2-004.patch, HADOOP-13171-branch-2-005.patch
>
>
> Add {{StorageStatistics}} support to S3A, collecting the same metrics as the instrumentation,
but sharing across all instances.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message