tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Wagner <wagner.mar...@gmail.com>
Subject A few questions on the APIs
Date Wed, 14 Aug 2013 04:28:33 GMT
Hey everyone,

I've been playing with the MRRSleep example to familiarize myself with
Tez. I've been trying to remove all the map and reduce parts to make
it "pure" Tez as an exercise, but I'm a bit hung up on the roles of
Processors and Tasks. It seems like they serve very similar roles. My
expectation was that Tasks would handle all the start-up and
coordination with the DAG AM, while the processors are more
user-facing and would mostly focus on the actual computation (given
that the processor can be specified via the DAG APIs). But it looks
like MRTask (which is ultimately extended as a Map or Reduce
Processor) does sends completion notifications to the AM with the
umbilical. Is there a good guideline as to what the responsibilities
of Processors and Tasks are and where the separation is?

Thanks for the insights,

View raw message