hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Íñigo Goiri (Jira) <j...@apache.org>
Subject [jira] [Assigned] (HADOOP-16947) Expired SampleStat should ignore when generating SlowPeersReport
Date Mon, 30 Mar 2020 14:24:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-16947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Íñigo Goiri reassigned HADOOP-16947:
------------------------------------

         Key: HADOOP-16947  (was: HDFS-14783)
    Assignee:     (was: Haibin Huang)
     Project: Hadoop Common  (was: Hadoop HDFS)

> Expired SampleStat should ignore when generating SlowPeersReport
> ----------------------------------------------------------------
>
>                 Key: HADOOP-16947
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16947
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Haibin Huang
>            Priority: Major
>         Attachments: HDFS-14783, HDFS-14783-001.patch, HDFS-14783-002.patch, HDFS-14783-003.patch,
HDFS-14783-004.patch, HDFS-14783-005.patch
>
>
> SlowPeersReport is generated by the SampleStat between tow dn, so it can present on nn's
jmx like this:
> {code:java}
> "SlowPeersReport" :[{"SlowNode":"dn2","ReportingNodes":["dn1"]}]
> {code}
> In each period, MutableRollingAverages will do a rollOverAvgs(), it will generate a
SumAndCount object which is based on SampleStat, and store it in a LinkedBlockingDeque<SumAndCount>,
the deque will be used to generate  SlowPeersReport. And the old member of deque won't be
removed until the queue is full. However, if dn1 don't send any packet to dn2 in the last
of 36*300_000 ms, the deque will be filled with an old member, because the number of last
SampleStat never change.I think these old SampleStats should be considered as expired message
and ignore them when generating a new SlowPeersReport.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message