tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xuan Cao <cao...@yahoo.com.INVALID>
Subject Tez statistics model
Date Tue, 06 Mar 2018 22:22:15 GMT

Hi,

 

Thisis Xuan with Microsoft. We are looking at the Tez statistics model trying toextend it
to support our custom workloads. Here are some observations of thefew statistics objects in
the current codebase:

1.   InputStatistics,OutputStatictics and VertexStatiscs are interfaces in tez-api.

2.   TaskStatistics is a classin tez-runtime-internals.

3.   VertexStatiscsImpl is aninner class of VertexImpl implementing VertexStatistics in
tez-dag.

4.   IOStatistics is in a classin tez-runtime-internals and IOStaticsticsImpl is a static
inner class ofVertexImpl extending IOStatistics but implementing InputStatistics andOutputStatistics.

 

Ourintention is to extend VertexStatisticsImpl so that we can aggregate the custompayload
in TaskStatistics. But the object model here seems to be ratherconfusing and not consistent.
We are planning to do some work in this area toimprove it, but not sure whether there is any
work going on right now and thestatus of it? 

 

Regards,

XuanCao




Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message