hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Trezzo (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-7250) Update Shared cache client api to use URLs
Date Mon, 25 Sep 2017 18:43:00 GMT
Chris Trezzo created YARN-7250:

             Summary: Update Shared cache client api to use URLs
                 Key: YARN-7250
                 URL: https://issues.apache.org/jira/browse/YARN-7250
             Project: Hadoop YARN
          Issue Type: Sub-task
            Reporter: Chris Trezzo
            Assignee: Chris Trezzo
            Priority: Minor

We should make the SharedCacheClient use api more consistent with other YARN api methods.
We can do this by doing two things:
# Update the SharedCacheClient#use api so that it returns a URL instead of a Path. Currently
yarn developers have to convert the path to a URL when creating a LocalResources. It would
be much smoother if they could just use a URL passed to them by the shared cache client.
# Remove the portion of the client that deals with fragments as this is not consistent with
the rest of YARN. This functionality is bleeding in from the MapReduce layer, which uses fragments
to keep track of destination file names. YARN's api does not use fragments. Instead  the ContainerLaunchContext
expects a Map<String, LocalResource> localResources, where the strings are the destination
file names. We should let the YARN application handle destination file names however it wants
instead of pushing this into the shared cache api. Additionally, fragments is a clunky way
to handle this.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

View raw message