hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Íñigo Goiri (Jira) <j...@apache.org>
Subject [jira] [Commented] (HADOOP-16947) Expired SampleStat should ignore when generating SlowPeersReport
Date Tue, 31 Mar 2020 15:13:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-16947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071877#comment-17071877
] 

Íñigo Goiri commented on HADOOP-16947:
--------------------------------------

Thanks [~huanghaibin] for the update.
We have already run the tests for HDFS, would you mind running the YARN tests too?
Yetus will only run commons tests now.
I don't know if any other project uses this class.

> Expired SampleStat should ignore when generating SlowPeersReport
> ----------------------------------------------------------------
>
>                 Key: HADOOP-16947
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16947
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Haibin Huang
>            Assignee: Haibin Huang
>            Priority: Major
>         Attachments: HADOOP-16947-001.patch, HADOOP-16947-002.patch, HDFS-14783, HDFS-14783-001.patch,
HDFS-14783-002.patch, HDFS-14783-003.patch, HDFS-14783-004.patch, HDFS-14783-005.patch
>
>
> SlowPeersReport is generated by the SampleStat between tow dn, so it can present on nn's
jmx like this:
> {code:java}
> "SlowPeersReport" :[{"SlowNode":"dn2","ReportingNodes":["dn1"]}]
> {code}
> In each period, MutableRollingAverages will do a rollOverAvgs(), it will generate a
SumAndCount object which is based on SampleStat, and store it in a LinkedBlockingDeque<SumAndCount>,
the deque will be used to generate  SlowPeersReport. And the old member of deque won't be
removed until the queue is full. However, if dn1 don't send any packet to dn2 in the last
of 36*300_000 ms, the deque will be filled with an old member, because the number of last
SampleStat never change.I think these old SampleStats should be considered as expired message
and ignore them when generating a new SlowPeersReport.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message