giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Reisman (JIRA)" <>
Subject [jira] [Commented] (GIRAPH-730) GiraphApplicationMaster race condition in resource loading
Date Fri, 23 Aug 2013 21:31:51 GMT


Eli Reisman commented on GIRAPH-730:

Hi Chaun,

I'm having a bit of trouble finding the concurrency issue here. The LOCAL_RESOURCES is a common
resource map that is initialized once per AM instance (which is hopefully a singleton at this
point!) and reused, unchanged, for each task launched. It is this object that is returned
from getTaskResourceMap to buildContainerLaunchContext, which returns the launch context,
populating the ContainerLaunchContext before any threads are run or submitted. from then on,
the method is just returning a reference to the same map each call.

If there is a concurrency issue, it might be more likely attributed to buildLaunchContainerContext.
But I'm not really seeing one.

If you are certain this is a concurrency issue and the syncronization fix is the only thing
verified to work, I'd try this:

I think the null check at the top of getTaskResourceMap is atomic by nature, you could just
add a syncronization block around the map construction portion. I think returning the unchanging
(and essentially immutable) map singleton in the loop of containers after that will be thread

> GiraphApplicationMaster race condition in resource loading
> ----------------------------------------------------------
>                 Key: GIRAPH-730
>                 URL:
>             Project: Giraph
>          Issue Type: Bug
>    Affects Versions: 1.0.0
>         Environment: Giraph with Yarn
>            Reporter: Chuan Lei
>            Assignee: Chuan Lei
>         Attachments: GIRAPH-730.v1.patch
> In, getTaskResourceMap function is not multi-thread safe,
which causes the application master fail to distribute the resources (jar, configuration file,
etc.) to each container.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message