flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8811) Add MiniClusterClient to allow fast MiniCluster operations
Date Thu, 01 Mar 2018 09:59:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16381771#comment-16381771
] 

ASF GitHub Bot commented on FLINK-8811:
---------------------------------------

Github user aljoscha commented on the issue:

    https://github.com/apache/flink/pull/5600
  
    +1 for merging this ASAP 👍 
    
    The error message seems a bit disconcerting, though:
    ```
    Finished org.apache.flink.test.checkpointing.HAIncrementalRocksDbBackendEventTimeWindowCheckpointingITCase#testSlidingTimeWindow.
    org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
    	at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$6.apply$mcV$sp(JobManager.scala:891)
    	at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$6.apply(JobManager.scala:834)
    	at org.apache.flink.runtime.jobmanager.JobManager$$anonfun$handleMessage$1$$anonfun$applyOrElse$6.apply(JobManager.scala:834)
    	at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
    	at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
    	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
    	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:415)
    	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
    Caused by: org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException:
Not enough free slots available to run the job. You can decrease the operator parallelism
or increase the number of slots per TaskManager in the configuration. Task to schedule: <
Attempt #3 (Source: Custom Source (1/1)) @ (unassigned) - [SCHEDULED] > with groupID <
bc764cd8ddf7a0cff126f51c16239658 > in sharing group < 7c540813226d407fbf248f58c16b372c
>. Resources available to scheduler: Number of instances=2, total number of slots=4, available
slots=0
    	at org.apache.flink.runtime.jobmanager.scheduler.Scheduler.scheduleTask(Scheduler.java:281)
    	at org.apache.flink.runtime.jobmanager.scheduler.Scheduler.allocateSlot(Scheduler.java:155)
    	at org.apache.flink.runtime.executiongraph.Execution.lambda$allocateAndAssignSlotForExecution$1(Execution.java:489)
    	at java.util.concurrent.CompletableFuture.uniComposeStage(CompletableFuture.java:981)
    	at java.util.concurrent.CompletableFuture.thenCompose(CompletableFuture.java:2124)
    	at org.apache.flink.runtime.executiongraph.Execution.allocateAndAssignSlotForExecution(Execution.java:487)
    	at org.apache.flink.runtime.executiongraph.ExecutionJobVertex.allocateResourcesForAll(ExecutionJobVertex.java:520)
    	at org.apache.flink.runtime.executiongraph.ExecutionGraph.scheduleEager(ExecutionGraph.java:957)
    	at org.apache.flink.runtime.executiongraph.ExecutionGraph.scheduleForExecution(ExecutionGraph.java:887)
    	at org.apache.flink.runtime.executiongraph.ExecutionGraph.restart(ExecutionGraph.java:1282)
    	at org.apache.flink.runtime.executiongraph.restart.ExecutionGraphRestartCallback.triggerFullRecovery(ExecutionGraphRestartCallback.java:59)
    	at org.apache.flink.runtime.executiongraph.restart.FixedDelayRestartStrategy$1.run(FixedDelayRestartStrategy.java:68)
    	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    	at java.lang.Thread.run(Thread.java:748)
    ```


> Add MiniClusterClient to allow fast MiniCluster operations
> ----------------------------------------------------------
>
>                 Key: FLINK-8811
>                 URL: https://issues.apache.org/jira/browse/FLINK-8811
>             Project: Flink
>          Issue Type: Improvement
>          Components: Tests
>    Affects Versions: 1.5.0, 1.6.0
>            Reporter: Till Rohrmann
>            Assignee: Till Rohrmann
>            Priority: Critical
>              Labels: flip-6
>             Fix For: 1.5.0, 1.6.0
>
>
> We should offer a {{ClusterClient}} implementation for the {{MiniCluster}}. That way
we would be able to submit and wait for result without polling how it would be the case by
using the {{RestClusterClient}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message