mrunit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan Chapin (JIRA)" <>
Subject [jira] [Commented] (MRUNIT-217) MapReduceDriver does not seem to work with MultipleOutput
Date Sun, 01 Feb 2015 04:28:34 GMT


Ryan Chapin commented on MRUNIT-217:

I am seeing the same behavior with 1.1.0 attempting to use the MapReduceDriver with a reducer
that is writing to multiple named outputs.

I have only stepped through the code once, so it wasn't exactly clear where the problem is.

As a work-around, instead of invoking the runTest() method on the MapReduceDriver instance
I invoked run() and then examined the counter values to validate my test.

Counters counters = mapReduceDriver.getCounters();
long actualCounter = counters.findCounter(Enum.key).getValue());

> MapReduceDriver does not seem to work with MultipleOutput
> ---------------------------------------------------------
>                 Key: MRUNIT-217
>                 URL:
>             Project: MRUnit
>          Issue Type: Question
>    Affects Versions: 1.1.0
>            Reporter: mathieu lavoie
> Hi,
> I'm having an issue running a test with org.apache.hadoop.mrunit.mapreduce.MapReduceDriver
using a reducer with multiple outputs. It seems like the generated results is always empty.
> It seems to work just fine for the ReduceDriver but not for the MapReduceDriver.
> The field:
> {noformat}
>   protected MockMultipleOutputs mos
> {noformat}
> Is set in the ReduceDriver using a mock but the MapReduceDriver is not aware of it. Therefore
the TestDriver thinks that there is no result.
> Am I missing something ?
> Here is a snippet of the configuration of my test:
> {noformat}
> @RunWith(PowerMockRunner.class)
> @PrepareForTest({StandardMetricsByListenerReducer.class})
> public class StandardMetricsByListenerMapReduceTest {
>     private MapReduceDriver<Object, Text, StandardMetricsDayKeyWritable, StandardMetricsValueWriteable,
StandardMetricsDayKeyWritable, StandardMetricsValueWriteable> mapReduceDriver;
>     @Before
>     public void setup() throws Exception {
>         StandardMetricsByListenerMapper mapper = new StandardMetricsByListenerMapper();
>         StandardMetricsByListenerReducer reducer = new StandardMetricsByListenerReducer();
>         mapReduceDriver = MapReduceDriver.newMapReduceDriver(mapper, reducer);
>     }
> }
> {noformat}
> Thanks

This message was sent by Atlassian JIRA

View raw message