hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mohak gupta <guptamo...@gmail.com>
Subject modify data distribution in jobconf
Date Sun, 01 Jan 2012 07:29:25 GMT

as part of my project I need to modify the data distribution layer in job
conf so as to achieve the following :

1) control which worker nodes should be  started based on the input data
given to them.

2) keep other worker nodes in some kind of sleep state.

3) based on the output emitted by the worker nodes and the data distributed
allow other worker nodes to start .

4) Perform this in a looping structure till the output is achieved.

basically I wish to control which worker nodes perform map and reduce
functions based on the data they have recieved.

Could you please help me by suggesting if this could be achieved and also
what are the tradeoffs involved, Any help is really appreciated

Mohak Gupta

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message