hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Radim Kolar (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-5153) Support for running combiners without reducers
Date Tue, 16 Apr 2013 15:41:15 GMT
Radim Kolar created MAPREDUCE-5153:

             Summary: Support for running combiners without reducers
                 Key: MAPREDUCE-5153
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5153
             Project: Hadoop Map/Reduce
          Issue Type: New Feature
            Reporter: Radim Kolar

scenario: Workflow mapper -> sort -> combiner -> hdfs

No api change is need, if user set combiner class and reducers = 0 then run combiner and sent
output to HDFS.

Popular libraries such as scalding and cascading are offering this functionality, but they
use caching entire mapper output in memory.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message