hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ivan Mitic (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10090) Jobtracker metrics not updated properly after execution of a mapreduce job
Date Fri, 15 Nov 2013 21:19:21 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13824105#comment-13824105

Ivan Mitic commented on HADOOP-10090:

Thanks Chris and Luke.

Luke, I tend to agree with the risk part.

I guess we have a couple of options on the table:
#1: We lock the source and use this to guarantee consistency of metrics reported thru sinks.
#2. We go with my first patch attached (I just have address the feedback). This will guarantee
that JMX returns back consistent state. Sinks on the other hand can get back partial results
(same as what we have today in branch-1).
#3. We go with my first patch + YARN-1043. This will address the problem from this Jira +
consistent results for sinks. 

Given our conversation from above, my preference is to go with #2. If we believe that YARN-1043
makes sense for branch-1, I would prefer to do it as a separate Jira.


It's definitely not easy to make the call... at least on my side... :)

PS. Shall we do a trunk patch too?
I believe we'll want to do a trunk patch too. Haven't tried to repro the problem there yet

> Jobtracker metrics not updated properly after execution of a mapreduce job
> --------------------------------------------------------------------------
>                 Key: HADOOP-10090
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10090
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: metrics
>    Affects Versions: 1.2.1
>            Reporter: Ivan Mitic
>            Assignee: Ivan Mitic
>         Attachments: HADOOP-10090.branch-1.2.patch, HADOOP-10090.branch-1.patch, OneBoxRepro.png
> After executing a wordcount mapreduce sample job, jobtracker metrics are not updated
properly. Often times the response from the jobtracker has higher number of job_completed
than job_submitted (for example 8 jobs completed and 7 jobs submitted). 
> Issue reported by Toma Paunovic.

This message was sent by Atlassian JIRA

View raw message