tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hitesh Shah <hit...@apache.org>
Subject TEZ-398: Changes to the underlying Task/Input/Output/Processor engine layer
Date Thu, 29 Aug 2013 23:58:02 GMT
Hi folks,

Siddharth, Bikas and I have been working on looking at how to re-factor/clean up the tez-engine
layer ( i.e the Input/Output/Processor and Task constructs ) to allow non-MR processors to
be built and allow easy introduction of new inputs and outputs. 

I posted an initial design/overview draft on TEZ-398 ( https://issues.apache.org/jira/secure/attachment/12600691/TEZ-398-Engine-Design.pdf
). Please feel free to add your comments to the jira. If you are interested in working on
any parts of the changes, comment on either the jira and/or mailing lists.  If there are folks
interested, we can set up a meeting where we can discuss the proposed design. 

Given that these changes are going to make the current implementation unstable for folks that
are starting to use to tez ( in hive/pig ), we plan to create a new branch ( likely to be
named TEZ-398 ) to work on these engine changes.

-- Hitesh 

View raw message