flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2797) CLI: Missing option to submit jobs in detached mode
Date Tue, 06 Oct 2015 18:11:26 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14945495#comment-14945495

ASF GitHub Bot commented on FLINK-2797:

Github user mxm commented on the pull request:

    @sachingoel0101 You're right. The detached YARN execution mode is broken for jobs which
use eager execution and build other jobs using the results. However, the detached YARN execution
could work if the ApplicationMaster executed the program. Essentially, this means shifting
the Client to the YARN cluster.
    @StephanEwen The way Sachin was going to handle the eager execution during detached mode
is to throw an Exception. However, the current code doesn't handle all cases. It is in the
nature of an interactive session that multiple jobs can be submitted. Results from submitted
jobs can be reused not only via count/collect but also using the returned JobExecutionResult.
Therefore, it is very difficult to deal with all these cases. IMHO we can only support non-interactive
detached job submissions.
    All in all, I think this is a nice idea but implementing it has too much overhead. The
implications of the changes are not quite clear. I pledge to change this to "Won't Fix" until
we think about a proper implementation. In the meantime, we should fix the detached YARN mode
or remove it.

> CLI: Missing option to submit jobs in detached mode
> ---------------------------------------------------
>                 Key: FLINK-2797
>                 URL: https://issues.apache.org/jira/browse/FLINK-2797
>             Project: Flink
>          Issue Type: Bug
>          Components: Command-line client
>    Affects Versions: 0.9, 0.10
>            Reporter: Maximilian Michels
>            Assignee: Sachin Goel
>             Fix For: 0.10
> Jobs can only be submitted in detached mode using YARN but not on a standalone installation.
This has been requested by users who want to submit a job, get the job id, and later query
its status.

This message was sent by Atlassian JIRA

View raw message