tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manu Zhang <owenzhang1...@gmail.com>
Subject In-memory shuffle in Tez
Date Mon, 21 Apr 2014 08:55:09 GMT

I used to develop on Tez-0.2.0 but haven't followed Tez-0.3.0 and
Tez-0.4.0.  I remeber Tez used to have InMemorySortedOutput and
InMemoryShuffleSorter which didn't spill map output to file.
I found they were removed in
TEZ-791<https://issues.apache.org/jira/browse/TEZ-791> so
I wonder whether Tez still supports some sort of In-memory shuffle ?
A related question is whether there are any examples for Ephemeral
EdgeProperty where a source task may stream its outputs directly to
destination task.

Manu Zhang

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message