tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kuhu Shukla (JIRA)" <j...@apache.org>
Subject [jira] [Created] (TEZ-3725) Cleanup http connections and other unnecessary fields in DAG Deletion tracker classes.
Date Fri, 12 May 2017 19:42:04 GMT
Kuhu Shukla created TEZ-3725:

             Summary: Cleanup http connections and other unnecessary fields in DAG Deletion
tracker classes.
                 Key: TEZ-3725
                 URL: https://issues.apache.org/jira/browse/TEZ-3725
             Project: Apache Tez
          Issue Type: Sub-task
            Reporter: Kuhu Shukla
            Assignee: Kuhu Shukla

JIRA to provides fixes for the following feedback comments from [~jlowe].

1. LocalContainerLauncher
I'm confused why we go through the trouble of serializing the hardcoded value of zero into
the aux service protocol buffer, stuff it into the env, then immediately go fetch it back
out and extract the integer from the byte buffer. Isn't this a complicated way to say, shufflePort
= 0?

2. tezDefaultComponentName only needs to be computed when cleanupDagDataOnComplete is true.
Actually may not be needed at all, see related comment for DagDeleteRunnable below.

3. DagDeleteRunnable
tezDefaultComponentName is unused? I think this transitively means pluginName is unused in
DeletionTracker which would simplify it's constructor signature.

4. DeletionTracker
Nit: addNodeShufflePorts method name being plural implies more than one port can be added
but it's only for adding a single node, port pair.

5. AMContainerHelpers
As Sidd mentioned before, we should avoid the redundant conf key lookup when creating each
container launch context.

6. DagDeleteRunnable
Do we need to do any cleanup on the httpConnection?

7. DeletionTrackerImpl
What if the submission to the executor throws RejectedExecutionException because the executor
was already shutdown and a late dagComplete was invoked?

This message was sent by Atlassian JIRA

View raw message