tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Seth (JIRA)" <j...@apache.org>
Subject [jira] [Created] (TEZ-3225) Include host port as top level fields in DataMovementEvents
Date Fri, 22 Apr 2016 06:50:12 GMT
Siddharth Seth created TEZ-3225:
-----------------------------------

             Summary: Include host port as top level fields in DataMovementEvents
                 Key: TEZ-3225
                 URL: https://issues.apache.org/jira/browse/TEZ-3225
             Project: Apache Tez
          Issue Type: Improvement
            Reporter: Siddharth Seth


Couple of steps for a small reduction in the payload size.
1. Include the host/port as top level fields. Allows for interned strings and Integers in
the AM and tasks, instead of having the same data encoded multiple times over in the byte
arrays.
2. Instead of the path being context.getUniqueIdentifer - individual fields like vertex_id,
task_id, attempt_id, output_id could be used - 4 integers/shorts instead of a long string.
This can be interpreted to have meaning and reconstructed by the consumer.

cc [~jeagles]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message