hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun Suresh <asur...@apache.org>
Subject [DISCUSS] AMRMClient::removeContainerRequest usage
Date Mon, 22 Aug 2016 19:42:11 GMT
Hello folks

I am currently working on refactoring some of the AMRMClient code for
YARN-4879.
I might have missed this but.. do we have any documentation on how the
AMRMClient should be used.

Specifically... Shouldn't it be mentioned somewhere that unless a user
explicitly calls the 'removeContainerRequest' method, the actual
'ContainerRequest' object will hang around in the AMRMClient. This will be
a problem for Long running application AMs that continually request
containers.

I understand that this is probably intentional, since the user of the
AMRMClient should probably call the 'getMatchingRequests' API which returns
the list of ContainerRequests associated with a returned Container and then
remove those ContainerRequests.

I see that the Tez AM does something similar via the
'TaskScheduler::deallocateTask()' once a container is allocated, but I do
not see it happen in the DistributedShell.

I feel either we should either document the need to remove satisfied
ContainerRequests by providing an example in the DistributedShell or via
explicit javadoc / documentation -or- somehow automate this in the
AMRMClient by examining the returned allocateResponse and removing matching
ContainerRequests.

Thoughts ?

Cheers
-Arun

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message