mrunit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brock Noland (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MRUNIT-129) Key object re-use in Reducer is inconsistent with MapReduce behaviour (new API)
Date Thu, 04 Apr 2013 14:53:16 GMT

    [ https://issues.apache.org/jira/browse/MRUNIT-129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13622387#comment-13622387
] 

Brock Noland commented on MRUNIT-129:
-------------------------------------

Ok, it's a big patch but most of it looks like tests. Unless someone else has time to review,
I'll give it a try next week.
                
> Key object re-use in Reducer is inconsistent with MapReduce behaviour (new API)
> -------------------------------------------------------------------------------
>
>                 Key: MRUNIT-129
>                 URL: https://issues.apache.org/jira/browse/MRUNIT-129
>             Project: MRUnit
>          Issue Type: Bug
>    Affects Versions: 0.9.0
>            Reporter: Dave Beech
>             Fix For: 1.1.0
>
>         Attachments: JIRA_129.patch, MRUNIT-129.unittest.patch
>
>
> Follow on issue from MRUNIT-127, also relating to grouping comparators. 
> In the new MapReduce API Reducer's reduce method, you are passed a reference to a key
and an Iterable for the values. What's not particularly clear is that as you iterate through
the values, the value of the key is also changing silently. This is not usually noticeable,
since the keys that were grouped often have the same value anyway, but with a custom grouping
comparator this isn't necessarily the case. 
> This is different to the old API, where you would get the first key in the group and
this wouldn't change as you iterate over the values. 
> MRUnit uses the "old API" style behaviour for both old and new APIs, so some unit tests
may not give a consistent result to an actual mapreduce job. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message