flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4272) Create a JobClient for job control and monitoring
Date Wed, 09 Nov 2016 13:49:59 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15650976#comment-15650976
] 

ASF GitHub Bot commented on FLINK-4272:
---------------------------------------

Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/2732#discussion_r87048964
  
    --- Diff: flink-clients/src/main/java/org/apache/flink/client/LocalExecutor.java ---
    @@ -172,29 +174,40 @@ public JobExecutionResult executePlan(Plan plan) throws Exception
{
     
     				// start the cluster for us
     				start();
    -			}
    -			else {
    +			} else {
     				// we use the existing session
     				shutDownAtEnd = false;
     			}
     
    -			try {
    -				Configuration configuration = this.flink.configuration();
    +			Configuration configuration = this.flink.configuration();
     
    -				Optimizer pc = new Optimizer(new DataStatistics(), configuration);
    -				OptimizedPlan op = pc.compile(plan);
    +			Optimizer pc = new Optimizer(new DataStatistics(), configuration);
    +			OptimizedPlan op = pc.compile(plan);
     
    -				JobGraphGenerator jgg = new JobGraphGenerator(configuration);
    -				JobGraph jobGraph = jgg.compileJobGraph(op, plan.getJobId());
    +			JobGraphGenerator jgg = new JobGraphGenerator(configuration);
    +			JobGraph jobGraph = jgg.compileJobGraph(op, plan.getJobId());
     
    -				boolean sysoutPrint = isPrintingStatusDuringExecution();
    -				return flink.submitJobAndWait(jobGraph, sysoutPrint);
    -			}
    -			finally {
    -				if (shutDownAtEnd) {
    -					stop();
    +			boolean sysoutPrint = isPrintingStatusDuringExecution();
    +
    +
    +			JobListeningContext jobListeningContext = flink.submitJob(jobGraph, sysoutPrint);
    +			JobClientEager jobClient = new JobClientEager(jobListeningContext);
    +
    +			Runnable cleanup = new Runnable() {
    +				@Override
    +				public void run() {
    +					if (shutDownAtEnd) {
    +						try {
    +							stop();
    +						} catch (Exception e) {
    +							throw new RuntimeException("Failed to run cleanup", e);
    --- End diff --
    
    This will crash the `JobClientEager` when calling `JobClientEager.shutdown`. Is this intended?


> Create a JobClient for job control and monitoring 
> --------------------------------------------------
>
>                 Key: FLINK-4272
>                 URL: https://issues.apache.org/jira/browse/FLINK-4272
>             Project: Flink
>          Issue Type: New Feature
>          Components: Client
>            Reporter: Maximilian Michels
>            Assignee: Maximilian Michels
>            Priority: Minor
>             Fix For: 1.2.0
>
>
> The aim of this new features is to expose a client to the user which allows to cancel
a running job, retrieve accumulators for a running job, or perform other actions in the future.
Let's call it {{JobClient}} for now (although this clashes with the existing JobClient class
which could be renamed to JobClientActorUtils instead).
> The new client should be returned from the {{ClusterClient}} class upon job submission.
The client should also be instantiatable by the users to retrieve the JobClient with a JobID.
> We should expose the new JobClient to the Java and Scala APIs using a new method on the
{{ExecutionEnvironment}} / {{StreamExecutionEnvironment}} called {{executeWithControl()}}
(perhaps we can find a better name).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message