hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Multiple resource requests for a given node (or all nodes)?
Date Sun, 11 Dec 2011 00:21:13 GMT
On Sat, Dec 10, 2011 at 12:23 PM, Patrick Wendell
<pwendell@eecs.berkeley.edu> wrote:

> What happens if an application wants to request multiple container
> types on a given node. E.g. say I need 10 2GB containers and 10 1GB
> containers, and I don't care which node they are on (i.e. RMNode.ANY).
> I really want to store 2 resource requests under RMNode.ANY in this
> case... don't I?
> Is the model just that an AM would ask for these in series?

My hunch is that this was overlooked because the resource sizes for MR
are basically set on a per-task-type level. That is, maps need X MB
and reduces need Y MB. Since maps and reduces are set at different
'priorities', they haven't conflicted.

Does it seem straightforward to change it to a multimap? Guava has a
nice implementation.

Todd Lipcon
Software Engineer, Cloudera

View raw message