flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4297) Yarn client can't determine fat jar location if path contains spaces
Date Mon, 01 Aug 2016 16:38:20 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15402373#comment-15402373
] 

ASF GitHub Bot commented on FLINK-4297:
---------------------------------------

Github user uce commented on the issue:

    https://github.com/apache/flink/pull/2320
  
    Tested this with Hadoop 2.4.1 and it works as expected with `bin/flink run -m yarn-cluster`:
    ```
    2016-08-01 16:32:06,214 INFO  org.apache.flink.yarn.Utils - Copying from 2016-08-01 16:32:06,214
INFO  org.apache.flink.yarn.Utils - Copying from file:/home/hadoop/mxm/flink/flink-dist/target/flink-1.1-SNAPSHOT-bin/flink
with spaces/conf/log4j.properties to gs://.../user/hadoop/.flink/application_1470041666362_0007/log4j.properties
    ...
    08/01/2016 16:33:24	Job execution switched to status FINISHED.
    ```
    
    But YARN sessions don't work. I get:
    ```
    $ bin/yarn-session.sh
    Error: Could not find or load main class with
    ```



> Yarn client can't determine fat jar location if path contains spaces
> --------------------------------------------------------------------
>
>                 Key: FLINK-4297
>                 URL: https://issues.apache.org/jira/browse/FLINK-4297
>             Project: Flink
>          Issue Type: Bug
>          Components: YARN Client
>            Reporter: Maximilian Michels
>            Assignee: Maximilian Michels
>             Fix For: 1.1.0, 1.2.0
>
>
> The code that automatically determines the fat jar path through the ProtectionDomain
of the Yarn class, receives a possibly URL encoded path string. We need to decode using the
system locale encoding, otherwise we can receive errors of the following when spaces are in
the file path: 
> {noformat}
> Caused by: java.io.FileNotFoundException: File file:/Users/max/Downloads/release-testing/flink-1.1.0-rc1/flink-1.1.0/build%20target/lib/flink-dist_2.11-1.1.0.jar
does not exist
>         at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:511)
>         at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:724)
>         at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:501)
>         at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:397)
>         at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
>         at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289)
>         at org.apache.hadoop.fs.LocalFileSystem.copyFromLocalFile(LocalFileSystem.java:82)
>         at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1836)
>         at org.apache.flink.yarn.Utils.setupLocalResource(Utils.java:129)
>         at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:616)
>         at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploy(AbstractYarnClusterDescriptor.java:365)
>         ... 6 more
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message