hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Evans <ev...@yahoo-inc.com>
Subject Re: Reg ChainReducer usage
Date Thu, 02 Jun 2011 14:38:51 GMT
Moving to mapreduce user.


The issue is with the shuffle.  The chain reducer cannot re-shuffle the output of a previous
reducer.  If you want that then you need to run a second reduce only job.  Instead usually
the chain reducer would have a single reducer followed by 0 or more mappers, that can process
the output of the reducer.


On 6/2/11 5:25 AM, "Ravi Teja" <raviteja@huawei.com> wrote:


I Had some queries in the usage of the ChainReducer .

1)Only one reducer can be set. If we try to set the second reducer to the
chain, IllegalArgumentException will be thrown. Then why is it a
chainreducer ?

2)We have a option chain.reducer.byValue where in, it will decide whether
the key value pair can be passed or not to the next Mapper/Reducer.
But why is this property significant, as only reducer is called at last in
the chain, no matter whatever the order is in the chain and there is nothing
to pass to.

Ravi Teja

This e-mail and attachments contain confidential information from HUAWEI,
which is intended only for the person or entity whose address is listed
above. Any use of the information contained herein in any way (including,
but not limited to, total or partial disclosure, reproduction, or
dissemination) by persons other than the intended recipient's) is
prohibited. If you receive this e-mail in error, please notify the sender by
phone or email immediately and delete it!

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message