hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erik Krogen (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-6918) ShuffleMetrics.ShuffleConnections Gauge Metric Climbs Infinitely
Date Tue, 25 Jul 2017 23:43:01 GMT
Erik Krogen created MAPREDUCE-6918:
--------------------------------------

             Summary: ShuffleMetrics.ShuffleConnections Gauge Metric Climbs Infinitely
                 Key: MAPREDUCE-6918
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6918
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: mrv2
            Reporter: Erik Krogen


We recently noticed that the {{mapred.ShuffleMetrics.ShuffleConnections}} metric seems to
climb infinitely, up to many millions (see attached graph), despite being supposedly a gauge
measure of the number of open connections:
{code:title=ShuffleHandler.java}
    @Metric("# of current shuffle connections")
        MutableGaugeInt shuffleConnections;
{code}

It seems that shuffleConnections gets incremented once for every map fetched, but only decremented
once for every request. It seems to me it should be modified to only be incremented once for
every request rather than for every map fetched, but I'm not familiar with the original intent.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org


Mime
View raw message