mrunit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bertrand Dechoux (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MRUNIT-158) withOutput doesn't generate useful information with MapWritable on error
Date Thu, 15 Nov 2012 08:50:15 GMT

    [ https://issues.apache.org/jira/browse/MRUNIT-158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13497849#comment-13497849
] 

Bertrand Dechoux commented on MRUNIT-158:
-----------------------------------------

Like I said, for MapWritable the toString()/equals()/hashcode() methods were not overrided
in the implementation.

This implies that
1) if the toString() outputs are different, then java will consider the MapWritable instances
as different. Only by reading the output of your test, you indeed know why it is failing.
(e1 =! 2059)
2) even if two MapWritable instances have exactly the same content (which implies that they
should be equals according the java Map contract), they won't in practice actually be equal

A solution for MRunit could be to provide our own equal implementation that should be used
only when the class of the key/value is an instance of MapWritable but that would means MRUnit
behaviour will differ from the standard Hadoop behaviour. Introducing such potential bugs
in a testing library is dodgy.

MRunit could also provide a custom toString() method using the same mechanism but that would
render the test less explicit when comparing two MapWritable instances with the same content.
The toString() output will be the same but that instances won't be equals. That would also
be quite confusing.

Of course, MRunit is an apache project, you are welcome to correct me. And if you have a solution,
you are also welcome to provide it or at least outline it.

For me, from the user point of view, the solution is to not use MapWritable. Sure, they are
handy and that would be great if they were actually respecting the java Map contract but that's
not the case as of now. And from a performance point of view, maps are not ideally because
you are transporting a (untyped) schema for each instance.
                
> withOutput doesn't generate useful information with MapWritable on error
> ------------------------------------------------------------------------
>
>                 Key: MRUNIT-158
>                 URL: https://issues.apache.org/jira/browse/MRUNIT-158
>             Project: MRUnit
>          Issue Type: Improvement
>    Affects Versions: 0.9.0
>            Reporter: Patrick Hunt
>            Priority: Minor
>
> Using withOutput generates the following, notice that the hashcode is printed rather
than some useful detail as to the content of the maps:
> Nov 9, 2012 2:40:01 PM org.apache.hadoop.mrunit.TestDriver logError
> SEVERE: Missing expected output (null-0, org.apache.hadoop.io.MapWritable@e1) at position
0.
> Nov 9, 2012 2:40:01 PM org.apache.hadoop.mrunit.TestDriver logError
> SEVERE: Received unexpected output (null-0, org.apache.hadoop.io.MapWritable@2059) at
position 0.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message