tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Or <andrewo...@gmail.com>
Subject General question about Tez shuffle
Date Fri, 19 Sep 2014 00:24:56 GMT
Hi all,

I have a general question of how Tez shuffle works on a yarn cluster.

I am familiar with how Yarn shuffle uses ShuffleHandler as an auxiliary
service of the node manager to serve spilled files. Does Tez use a similar
interface? I am running an HDP 2.1 cluster with Tez installed, and I
noticed that my "yarn.nodemanager.aux-services" does not include anything
related to Tez. Does Tez run its own HTTP server to serve these files? If
so, are these servers per-node or per-application?

Thanks a lot in advance,
Andrew

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message