mrunit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bertrand Dechoux (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (MRUNIT-158) withOutput doesn't generate useful information with MapWritable on error
Date Fri, 09 Nov 2012 23:17:12 GMT

    [ https://issues.apache.org/jira/browse/MRUNIT-158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13494410#comment-13494410
] 

Bertrand Dechoux edited comment on MRUNIT-158 at 11/9/12 11:15 PM:
-------------------------------------------------------------------

I wouldn't put that as critical. I am not even sure that doing anything for that subject would
be a good thing.

Of course, a user friendly support of map would be nice but the underlying issue is with Haddop
implementation of MapWritable. As of 1.03, there is no (overriding) implementation of toString(),
equals() and hascode() methods. (see HADOOP-7153)

MRunit could detect the type being a MapWritable and in that case use its own definition of
toString(), equals() and hashcode() but that would be a feature and not a bug fixe and I would
not really be favorable to it.

Maps are indeed nice to transfer 'dynamic data' but it implies that each object carries its
own schema so that's not very efficient. You may want to look at tuples instead (and at library
such as Cascading.)


                
      was (Author: bdechoux):
    I wouldn't put that as critical. I am not even sure that doing anything for that subject
would be a good thing.

Of course, a user friendly support of map would be nice but the underlying issue is with Haddop
implementation of MapWritable. As of 1.03, there is no (overriding) implementation of toString(),
equals() and hascode() methods. (see HADOOP-7153)

MRunit could detect the type being a MapWritable and in that case use its own definition of
toString(), equals() and hashcode() but that would be a feature and not a bug fixe and I would
not really be favorable to it.

Maps are indeed nice to transfer 'dynamic data' but it implies that each object carries its
own typeless schema so that's not very efficient. You may want to look at tuples instead (and
at library such as Cascading.)


                  
> withOutput doesn't generate useful information with MapWritable on error
> ------------------------------------------------------------------------
>
>                 Key: MRUNIT-158
>                 URL: https://issues.apache.org/jira/browse/MRUNIT-158
>             Project: MRUnit
>          Issue Type: Improvement
>    Affects Versions: 0.9.0
>            Reporter: Patrick Hunt
>            Priority: Minor
>
> Using withOutput generates the following, notice that the hashcode is printed rather
than some useful detail as to the content of the maps:
> Nov 9, 2012 2:40:01 PM org.apache.hadoop.mrunit.TestDriver logError
> SEVERE: Missing expected output (null-0, org.apache.hadoop.io.MapWritable@e1) at position
0.
> Nov 9, 2012 2:40:01 PM org.apache.hadoop.mrunit.TestDriver logError
> SEVERE: Received unexpected output (null-0, org.apache.hadoop.io.MapWritable@2059) at
position 0.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message