tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhiyuan Yang (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (TEZ-3848) Tez Local mode doesn't localize distributed cache files
Date Fri, 05 Jan 2018 00:18:24 GMT

     [ https://issues.apache.org/jira/browse/TEZ-3848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Zhiyuan Yang closed TEZ-3848.

> Tez Local mode doesn't localize distributed cache files
> -------------------------------------------------------
>                 Key: TEZ-3848
>                 URL: https://issues.apache.org/jira/browse/TEZ-3848
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Jacob Tolar
>            Assignee: Jacob Tolar
>             Fix For: 0.9.1
>         Attachments: TEZ-3848.1.patch
> Tez doesn't symlink LocalResources into place in LocalContainerLauncher.
> In YARN mode, Yarn takes care of this when it launches the container. But in local mode,
if you're depending on a file existing in the distributed cache, it's never symlinked into
place (so you're out of luck).
> We test our pig scripts in local mode and have some tools to set up the distributed cache
the same way it would work in production. This works fine in Mapreduce mode but are unable
to use Pig + Tez local mode for testing due to this problem.
> I have a fix working and will submit a PR once I rebase it.
> [~jeagles] [~wlauer@yahoo-inc.com]

This message was sent by Atlassian JIRA

View raw message