flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2797) CLI: Missing option to submit jobs in detached mode
Date Fri, 09 Oct 2015 15:22:27 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14950553#comment-14950553

ASF GitHub Bot commented on FLINK-2797:

Github user sachingoel0101 commented on the pull request:

    That won't suffice. Even if we allow one execute call, we cannot be sure that user won't
try to access the `JobExecutionResult`. That also is an issue.
    Disallowing every program which doesn't implement the `Program` interface isn't a good
solution. I'm not an expert but most people would usually have a main method. In that case,
it doesn't make sense to have a detached mode if we cannot allow such executions.
    We might be able to detect the exact program structure by exploiting the Context Environment.
Before actually executing the program, we instead first do a validation.
    1. On an execute call, instead of actually executing the program, we instead build the
plan and store it. If another call to execute comes in, we fail.
    2. The `JobExecutionResult` can be wrapped in a `DetachedExecutionResult` which will fail
if user tries to access, say accumulators.
    In the documentation for detached mode, we should specify that only programs with one
execute call, and no usage of the execution result will be allowed. This handles the eager
execution issue also.
    But I'm concerned this isn't the best possible design.

> CLI: Missing option to submit jobs in detached mode
> ---------------------------------------------------
>                 Key: FLINK-2797
>                 URL: https://issues.apache.org/jira/browse/FLINK-2797
>             Project: Flink
>          Issue Type: Bug
>          Components: Command-line client
>    Affects Versions: 0.9, 0.10
>            Reporter: Maximilian Michels
>            Assignee: Sachin Goel
>             Fix For: 0.10
> Jobs can only be submitted in detached mode using YARN but not on a standalone installation.
This has been requested by users who want to submit a job, get the job id, and later query
its status.

This message was sent by Atlassian JIRA

View raw message