mrunit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan Chapin (JIRA)" <>
Subject [jira] [Commented] (MRUNIT-217) MapReduceDriver does not seem to work with MultipleOutput
Date Fri, 20 Nov 2015 04:51:11 GMT


Ryan Chapin commented on MRUNIT-217:

Attaching patch.

The basics of this fix is updating the signature of the run method in the TestDriver type
to return a data container object that can contain the MultipleOutputs actual results from
the Reducer under test.  Otherwise, that data is never in scope and cannot be validated against
the expected values.

I also noticed that the org.mockito.mockito-core dependency was marked <optional>true</optional>
and that caused a class loading failure in an existing project that I was using.  I created
a test harness to do some integration testing of the MRUnit project (
and verified that that change in the pom would break upgrading from 1.1.0 to 1.2.0 when attempting
to test the MultipleOutputs class.  By commenting out the <optional> tag in that dependency
I was able to get the 1.2.0-SNAPSHOT version to run in the test harness and an existing project.
 This patch has that tag commented out in the pom.

Please let me know if anyone has any questions.

> MapReduceDriver does not seem to work with MultipleOutput
> ---------------------------------------------------------
>                 Key: MRUNIT-217
>                 URL:
>             Project: MRUnit
>          Issue Type: Question
>    Affects Versions: 1.1.0
>            Reporter: mathieu lavoie
>         Attachments: MRUNIT-217.patch
> Hi,
> I'm having an issue running a test with org.apache.hadoop.mrunit.mapreduce.MapReduceDriver
using a reducer with multiple outputs. It seems like the generated results is always empty.
> It seems to work just fine for the ReduceDriver but not for the MapReduceDriver.
> The field:
> {noformat}
>   protected MockMultipleOutputs mos
> {noformat}
> Is set in the ReduceDriver using a mock but the MapReduceDriver is not aware of it. Therefore
the TestDriver thinks that there is no result.
> Am I missing something ?
> Here is a snippet of the configuration of my test:
> {noformat}
> @RunWith(PowerMockRunner.class)
> @PrepareForTest({StandardMetricsByListenerReducer.class})
> public class StandardMetricsByListenerMapReduceTest {
>     private MapReduceDriver<Object, Text, StandardMetricsDayKeyWritable, StandardMetricsValueWriteable,
StandardMetricsDayKeyWritable, StandardMetricsValueWriteable> mapReduceDriver;
>     @Before
>     public void setup() throws Exception {
>         StandardMetricsByListenerMapper mapper = new StandardMetricsByListenerMapper();
>         StandardMetricsByListenerReducer reducer = new StandardMetricsByListenerReducer();
>         mapReduceDriver = MapReduceDriver.newMapReduceDriver(mapper, reducer);
>     }
> }
> {noformat}
> Thanks

This message was sent by Atlassian JIRA

View raw message