hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14425) Add more s3guard metrics
Date Thu, 25 May 2017 12:41:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16024676#comment-16024676
] 

Steve Loughran commented on HADOOP-14425:
-----------------------------------------

Latest dump of stats from a committer test run. Note how the latency values are in a different
case from the others, including the s3guard_ props. They should be downcased
{code}
2017-05-25 12:24:08,466 [ScalaTest-main-running-S3ACommitDataframeSuite] INFO  s3.S3AOperations
(CloudLogging.scala:logInfo(56)) -   S3guard_metadatastore_put_path_latency50thPercentileLatency
=  0
  S3guard_metadatastore_put_path_latency75thPercentileLatency =  0
  S3guard_metadatastore_put_path_latency90thPercentileLatency =  0
  S3guard_metadatastore_put_path_latency95thPercentileLatency =  0
  S3guard_metadatastore_put_path_latency99thPercentileLatency =  0
  S3guard_metadatastore_put_path_latencyNumOps =  0
  committer_bytes_committed =  314
  committer_commits_aborted =  0
  committer_commits_completed =  1
  committer_commits_created =  1
  committer_commits_failed =  0
  committer_commits_reverted =  0
  committer_jobs_completed =  1
  committer_jobs_failed =  0
  committer_tasks_completed =  1
  committer_tasks_failed =  0
  directories_created =  0
  directories_deleted =  0
  fake_directories_deleted =  4
  files_copied =  0
  files_copied_bytes =  0
  files_created =  0
  files_deleted =  2
  ignored_errors =  1
  object_continue_list_requests =  0
  object_copy_requests =  0
  object_delete_requests =  2
  object_list_requests =  5
  object_metadata_requests =  8
  object_multipart_aborted =  0
  object_put_bytes =  314
  object_put_bytes_pending =  0
  object_put_requests =  1
  object_put_requests_active =  0
  object_put_requests_completed =  1
  op_copy_from_local_file =  0
  op_exists =  1
  op_get_file_status =  3
  op_glob_status =  0
  op_is_directory =  0
  op_is_file =  0
  op_list_files =  0
  op_list_located_status =  0
  op_list_status =  0
  op_mkdirs =  0
  op_rename =  0
  s3guard_metadatastore_initialization =  0
  s3guard_metadatastore_put_path_request =  1
  stream_aborted =  0
  stream_backward_seek_operations =  0
  stream_bytes_backwards_on_seek =  0
  stream_bytes_discarded_in_abort =  0
  stream_bytes_read =  0
  stream_bytes_read_in_close =  0
  stream_bytes_skipped_on_seek =  0
  stream_close_operations =  0
  stream_closed =  0
  stream_forward_seek_operations =  0
  stream_opened =  0
  stream_read_exceptions =  0
  stream_read_fully_operations =  0
  stream_read_operations =  0
  stream_read_operations_incomplete =  0
  stream_seek_operations =  0
  stream_write_block_uploads =  0
  stream_write_block_uploads_aborted =  0
  stream_write_block_uploads_active =  0
  stream_write_block_uploads_committed =  0
  stream_write_block_uploads_data_pending =  0
  stream_write_block_uploads_pending =  0
  stream_write_failures =  0
  stream_write_total_data =  0
  stream_write_total_time =  0
{code}


> Add more s3guard metrics
> ------------------------
>
>                 Key: HADOOP-14425
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14425
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Ai Deng
>
> The metrics suggested to add:
> Status:
> S3GUARD_METADATASTORE_ENABLED
> S3GUARD_METADATASTORE_IS_AUTHORITATIVE
> Operations:
> S3GUARD_METADATASTORE_INITIALIZATION
> S3GUARD_METADATASTORE_DELETE_PATH
> S3GUARD_METADATASTORE_DELETE_PATH_LATENCY
> S3GUARD_METADATASTORE_DELETE_SUBTREE_PATCH
> S3GUARD_METADATASTORE_GET_PATH
> S3GUARD_METADATASTORE_GET_PATH_LATENCY
> S3GUARD_METADATASTORE_GET_CHILDREN_PATH
> S3GUARD_METADATASTORE_GET_CHILDREN_PATH_LATENCY
> S3GUARD_METADATASTORE_MOVE_PATH
> S3GUARD_METADATASTORE_PUT_PATH
> S3GUARD_METADATASTORE_PUT_PATH_LATENCY
> S3GUARD_METADATASTORE_CLOSE
> S3GUARD_METADATASTORE_DESTORY
> From S3Guard:
> S3GUARD_METADATASTORE_MERGE_DIRECTORY
> For the failures:
> S3GUARD_METADATASTORE_DELETE_FAILURE
> S3GUARD_METADATASTORE_GET_FAILURE
> S3GUARD_METADATASTORE_PUT_FAILURE
> Etc:
> S3GUARD_METADATASTORE_PUT_RETRY_TIMES



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message