flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8769) Quickstart job execution in IDE logs contain several exceptions
Date Thu, 01 Mar 2018 13:02:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16381957#comment-16381957
] 

ASF GitHub Bot commented on FLINK-8769:
---------------------------------------

GitHub user NicoK opened a pull request:

    https://github.com/apache/flink/pull/5611

    [FLINK-8769][flip6] do not print error causing exceptions without debugging

    ## What is the purpose of the change
    
    Quickstart job execution in IDE logs these two exceptions which may mislead the user in
thinking something went wrong when indeed it did not:
    
    1)
    ```
    10:23:18,413 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Failed
to load web based job submission extension.
    org.apache.flink.util.FlinkException: The module flink-runtime-web could not be found
in the class path. Please add this jar in order to enable web based job submission.
    	at org.apache.flink.runtime.webmonitor.WebMonitorUtils.loadWebSubmissionExtension(WebMonitorUtils.java:255)
    	at org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint.initializeHandlers(DispatcherRestEndpoint.java:111)
    	at org.apache.flink.runtime.rest.RestServerEndpoint.start(RestServerEndpoint.java:127)
    	at org.apache.flink.runtime.minicluster.MiniCluster.start(MiniCluster.java:330)
    	at org.apache.flink.streaming.api.environment.Flip6LocalStreamEnvironment.execute(Flip6LocalStreamEnvironment.java:101)
    	at flink.test.StreamingJob.main(StreamingJob.java:64)
    ```
    2)
    ```
    10:23:19,099 INFO  org.apache.flink.runtime.taskexecutor.TaskExecutor            - Close
ResourceManager connection 3ebc4874f7a952d9ba4e4fc6ac574b89.
    org.apache.flink.util.FlinkException: New ResourceManager leader found under: null(null)
    	at org.apache.flink.runtime.taskexecutor.TaskExecutor.notifyOfNewResourceManagerLeader(TaskExecutor.java:851)
    	at org.apache.flink.runtime.taskexecutor.TaskExecutor.access$900(TaskExecutor.java:127)
    	at org.apache.flink.runtime.taskexecutor.TaskExecutor$ResourceManagerLeaderListener.lambda$notifyLeaderAddress$0(TaskExecutor.java:1359)
    	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:295)
    	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:150)
    	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$onReceive$1(AkkaRpcActor.java:132)
    	at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:544)
    	at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
    	at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)
    	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
    	at akka.actor.ActorCell.invoke(ActorCell.scala:495)
    	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
    	at akka.dispatch.Mailbox.run(Mailbox.scala:224)
    	at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
    	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
    ```
    
    While for debugging they may be useful, for normal operation it is enough to print the
messages
    themselves, especially since some more details were already logged before them.
    
    ## Brief change log
    
    - only print error causing exceptions in `DispatcherRestEndpoint` and `TaskExecutor` when
in debug mode similar to how queryable state loading is done
    
    ## Verifying this change
    
    This change can be verified as follows:
    - started a fresh quickstart project with a simple streaming program like `env.fromElements(1,2,3,4,5).print();`
in IntelliJ and verified there are no exceptions in the logs
    
    ## Does this pull request potentially affect one of the following parts:
    
      - Dependencies (does it add or upgrade a dependency): **no**
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`:  **no**
      - The serializers:  **no**
      - The runtime per-record code paths (performance sensitive):  **no**
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing,
Yarn/Mesos, ZooKeeper: **yes**
      - The S3 file system connector:  **no**
    
    ## Documentation
    
      - Does this pull request introduce a new feature?  **no**
      - If yes, how is the feature documented? **not applicable**


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/NicoK/flink flink-8769

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/5611.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5611
    
----
commit d4e55e6a859c2b14edaab89797416b086fe609b6
Author: Nico Kruber <nico@...>
Date:   2018-03-01T12:53:21Z

    [FLINK-8769][flip6] do not print error causing exceptions without debugging
    
    In DispatcherRestEndpoint and TaskExecutor, there were two places where without
    errors (running a job inside an IDE) exceptions were logged. While for debugging
    they may be useful, for normal operation it is enough to print the messages
    themselves, especially since some more details were already logged before.

----


> Quickstart job execution in IDE logs contain several exceptions
> ---------------------------------------------------------------
>
>                 Key: FLINK-8769
>                 URL: https://issues.apache.org/jira/browse/FLINK-8769
>             Project: Flink
>          Issue Type: Bug
>          Components: Local Runtime
>    Affects Versions: 1.5.0
>            Reporter: Chesnay Schepler
>            Assignee: Nico Kruber
>            Priority: Blocker
>             Fix For: 1.5.0
>
>
> While checking out [the PR for FLINK-8761|https://github.com/apache/flink/pull/5569]
and running a job in the IDE several exceptions are being logged. The job still runs properly
though.
> {code:java}
> ...
> 14:19:52,546 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Failed
to load web based job submission extension.
> org.apache.flink.util.FlinkException: Could not load web submission extension.
> 	at org.apache.flink.runtime.webmonitor.WebMonitorUtils.loadWebSubmissionExtension(WebMonitorUtils.java:252)
> 	at org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint.initializeHandlers(DispatcherRestEndpoint.java:111)
> 	at org.apache.flink.runtime.rest.RestServerEndpoint.start(RestServerEndpoint.java:124)
> 	at org.apache.flink.runtime.minicluster.MiniCluster.start(MiniCluster.java:320)
> 	at org.apache.flink.client.LocalExecutor.createJobExecutorService(LocalExecutor.java:144)
> 	at org.apache.flink.client.LocalExecutor.start(LocalExecutor.java:118)
> 	at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:212)
> 	at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
> 	at org.apache.flink.api.java.ExecutionEnvironment.execute(ExecutionEnvironment.java:815)
> 	at org.apache.flink.api.java.DataSet.collect(DataSet.java:413)
> 	at org.apache.flink.api.java.DataSet.print(DataSet.java:1652)
> 	at iqst.BatchJob.main(BatchJob.java:39)
> Caused by: java.lang.reflect.InvocationTargetException
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> 	at org.apache.flink.runtime.webmonitor.WebMonitorUtils.loadWebSubmissionExtension(WebMonitorUtils.java:243)
> 	... 11 more
> Caused by: org.apache.flink.util.ConfigurationException: Config parameter 'Key: 'jobmanager.rpc.address'
, default: null (deprecated keys: [])' is missing (hostname/address of JobManager to connect
to).
> 	at org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.getJobManagerAddress(HighAvailabilityServicesUtils.java:137)
> 	at org.apache.flink.runtime.highavailability.HighAvailabilityServicesUtils.createHighAvailabilityServices(HighAvailabilityServicesUtils.java:79)
> 	at org.apache.flink.client.program.ClusterClient.<init>(ClusterClient.java:148)
> 	at org.apache.flink.client.program.rest.RestClusterClient.<init>(RestClusterClient.java:144)
> 	at org.apache.flink.client.program.rest.RestClusterClient.<init>(RestClusterClient.java:135)
> 	at org.apache.flink.runtime.webmonitor.WebSubmissionExtension.<init>(WebSubmissionExtension.java:61)
> 	... 16 more
> 14:19:53,140 INFO  org.apache.flink.runtime.dispatcher.DispatcherRestEndpoint    - Rest
endpoint listening at 127.0.0.1:64908
> ...
> 14:19:56,546 INFO  org.apache.flink.runtime.taskexecutor.TaskExecutor            - Close
ResourceManager connection b8a2cff59ba07813067a64ebaf7d7889.
> org.apache.flink.util.FlinkException: New ResourceManager leader found under: null(null)
> 	at org.apache.flink.runtime.taskexecutor.TaskExecutor.notifyOfNewResourceManagerLeader(TaskExecutor.java:853)
> 	at org.apache.flink.runtime.taskexecutor.TaskExecutor.access$900(TaskExecutor.java:127)
> 	at org.apache.flink.runtime.taskexecutor.TaskExecutor$ResourceManagerLeaderListener.lambda$notifyLeaderAddress$0(TaskExecutor.java:1359)
> 	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:292)
> 	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:147)
> 	at org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$onReceive$0(AkkaRpcActor.java:129)
> 	at akka.actor.ActorCell$$anonfun$become$1.applyOrElse(ActorCell.scala:544)
> 	at akka.actor.Actor$class.aroundReceive(Actor.scala:502)
> 	at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:95)
> 	at akka.actor.ActorCell.receiveMessage(ActorCell.scala:526)
> 	at akka.actor.ActorCell.invoke(ActorCell.scala:495)
> 	at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)
> 	at akka.dispatch.Mailbox.run(Mailbox.scala:224)
> 	at akka.dispatch.Mailbox.exec(Mailbox.scala:234)
> 	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> 	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> 	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> 	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> ...
> java.lang.Exception: Leader election service is shutting down
> 	at org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService.shutdown(EmbeddedLeaderService.java:93)
> 	at org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedHaServices.close(EmbeddedHaServices.java:152)
> 	at org.apache.flink.runtime.highavailability.nonha.AbstractNonHaServices.closeAndCleanupAllData(AbstractNonHaServices.java:110)
> 	at org.apache.flink.runtime.minicluster.MiniCluster.shutdownInternally(MiniCluster.java:498)
> 	at org.apache.flink.runtime.minicluster.MiniCluster.shutdown(MiniCluster.java:380)
> 	at org.apache.flink.runtime.minicluster.MiniCluster.closeAsync(MiniCluster.java:790)
> 	at org.apache.flink.util.AutoCloseableAsync.close(AutoCloseableAsync.java:40)
> 	at org.apache.flink.client.LocalExecutor.stop(LocalExecutor.java:163)
> 	at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:235)
> 	at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
> 	at org.apache.flink.api.java.ExecutionEnvironment.execute(ExecutionEnvironment.java:815)
> 	at org.apache.flink.api.java.DataSet.collect(DataSet.java:413)
> 	at org.apache.flink.api.java.DataSet.print(DataSet.java:1652)
> 	at iqst.BatchJob.main(BatchJob.java:39)
> 14:19:57,450 ERROR org.apache.flink.runtime.minicluster.MiniCluster              - TaskManager
#0 failed.
> java.lang.Exception: Leader election service is shutting down
> 	at org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedLeaderService.shutdown(EmbeddedLeaderService.java:93)
> 	at org.apache.flink.runtime.highavailability.nonha.embedded.EmbeddedHaServices.close(EmbeddedHaServices.java:152)
> 	at org.apache.flink.runtime.highavailability.nonha.AbstractNonHaServices.closeAndCleanupAllData(AbstractNonHaServices.java:110)
> 	at org.apache.flink.runtime.minicluster.MiniCluster.shutdownInternally(MiniCluster.java:498)
> 	at org.apache.flink.runtime.minicluster.MiniCluster.shutdown(MiniCluster.java:380)
> 	at org.apache.flink.runtime.minicluster.MiniCluster.closeAsync(MiniCluster.java:790)
> 	at org.apache.flink.util.AutoCloseableAsync.close(AutoCloseableAsync.java:40)
> 	at org.apache.flink.client.LocalExecutor.stop(LocalExecutor.java:163)
> 	at org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:235)
> 	at org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:91)
> 	at org.apache.flink.api.java.ExecutionEnvironment.execute(ExecutionEnvironment.java:815)
> 	at org.apache.flink.api.java.DataSet.collect(DataSet.java:413)
> 	at org.apache.flink.api.java.DataSet.print(DataSet.java:1652)
> 	at iqst.BatchJob.main(BatchJob.java:39)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message