Though I do agree with the others statements that the comparator should have it's own unit tests outside of MRUnit. Using it in MRUnit is more of an integration tests for the comparator.


On Wed, Jan 15, 2014 at 10:00 AM, Brock Noland <brock@cloudera.com> wrote:
Can you send me a full code example that reproduces the error?

I believe you are using KeyOrderComparator correctly.

Brock


On Tue, Jan 14, 2014 at 11:29 AM, alex bohr <alexjbohr@gmail.com> wrote:
Thanks guys!
I did end up writing my own serializer (wasn't as complicated as I thought) but Tom White's code, of course, is cleaner.  That example will help.

So can anyone explain a use case for the method "setKeyOrderComparator" in the mapReduceDriver class?  
I'm not understanding when/why I would use that method in a test case.

Thanks again for the advice, much appreciated!

ps.  Totally different topic I did run some tests yesterday of the full job on our cluster with the raw comparator vs the old full-object comparator and the old comparator ran slightly faster...?!?  I'm going to blame it on the cluster being very busy at the time of testing.


On Tue, Jan 14, 2014 at 7:19 AM, Tom Wheeler <twheeler@cloudera.com> wrote:
On Tue, Jan 14, 2014 at 3:51 AM, Bertrand Dechoux <dechouxb@gmail.com> wrote:
> I would also tend to say that the best would be to test the comparator
> separately.

It's pretty straightforward to do this and is also the approach that
Tom White took in Hadoop:The Definitive Guide. Here's the relevant
example from his GitHub repo:

   https://github.com/tomwhite/hadoop-book/blob/master/ch04/src/test/java/TextPairTest.java

The utility methods he uses to convert a Writable instance to/from an
array of bytes are straightforward::

   https://github.com/tomwhite/hadoop-book/blob/master/ch04/src/test/java/WritableTestBase.java

Hope that helps,

Tom Wheeler




--
Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org



--
Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org