mrunit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Francke (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MRUNIT-13) Add support for MultipleOutputs
Date Mon, 21 Jul 2014 19:54:40 GMT

    [ https://issues.apache.org/jira/browse/MRUNIT-13?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069165#comment-14069165
] 

Lars Francke commented on MRUNIT-13:
------------------------------------

I just tried again. This reproduces the issue:

{code}
import java.io.IOException;

import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.lib.output.MultipleOutputs;
import org.apache.hadoop.mrunit.mapreduce.MapDriver;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.powermock.core.classloader.annotations.PrepareForTest;
import org.powermock.modules.junit4.PowerMockRunner;

@RunWith(PowerMockRunner.class)
@PrepareForTest(TestDemo.TestMapper.class)
public class TestDemo {

  @Test
  public void testMrunit() throws IOException {
    TestMapper mapper = new TestMapper();
    MapDriver<Text, Text, Text, Text> mapDriver = new MapDriver<Text, Text, Text,
Text>();
    mapDriver.setMapper(mapper);
    mapDriver.withInput(new Text("foo"), new Text("bar"))
      .withMultiOutput("foobar", new Text("foo"), new Text("bar"))
      .runTest();
  }

  public static class TestMapper extends Mapper<Text, Text, Text, Text> {

    private MultipleOutputs<Text, Text> mos;

    @Override
    protected void setup(Context context) throws IOException, InterruptedException {
      mos = new MultipleOutputs<Text, Text>(context);
    }

    @Override
    public void map(Text key, Text value, Context context) throws IOException, InterruptedException
{
      mos.write("foobar", new Text("foo"), new Text("bar"));
    }

    @Override
    protected void cleanup(Context context) throws IOException, InterruptedException {
      mos.close();
    }
  }

}
{code}

The Mapper using the MultipleOutputs format needs to be prepared otherwise the named outputs
are not known.

> Add support for MultipleOutputs
> -------------------------------
>
>                 Key: MRUNIT-13
>                 URL: https://issues.apache.org/jira/browse/MRUNIT-13
>             Project: MRUnit
>          Issue Type: New Feature
>            Reporter: E. Sammer
>            Assignee: Jim Donofrio
>             Fix For: 1.1.0
>
>         Attachments: MRUNIT-13.patch, mapreduce.zip, vcs-diff-8290140541471322425-2.patch,
vcs-diff8290140541471322425.patch
>
>
> Add support to mrunit for Hadoop's MultipleOutputs.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message