flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gary Wu (Jira)" <j...@apache.org>
Subject [jira] [Updated] (FLINK-22688) Root Exception can not be shown on Web UI in Flink 1.13.0
Date Thu, 20 May 2021 12:42:00 GMT

     [ https://issues.apache.org/jira/browse/FLINK-22688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Gary Wu updated FLINK-22688:
----------------------------
    Attachment: taskmanager_log_v1.txt

> Root Exception can not be shown on Web UI in Flink 1.13.0
> ---------------------------------------------------------
>
>                 Key: FLINK-22688
>                 URL: https://issues.apache.org/jira/browse/FLINK-22688
>             Project: Flink
>          Issue Type: Bug
>          Components: Runtime / Coordination
>    Affects Versions: 1.13.0
>            Reporter: Gary Wu
>            Assignee: Matthias
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 1.13.1
>
>         Attachments: jobmanager_log_v1.txt.zip, taskmanager_log_v1.txt
>
>
> Hi,
>  
> We have upgraded our Flink applications to 1.13.0 but we found that Root Exception can
not be shown on Web UI with an internal server error message. After opening a browser development
console and trace the message, we found that there is an exception in job manager:
>  
> _{color:#000000}2021-05-12 13:30:45,589 ERROR org.apache.flink.runtime.rest.handler.job.JobExceptionsHandler
[] - Unhandled exception.{color}_
> _{color:#000000}java.lang.IllegalArgumentException: The location must not be null for
a non-global failure.{color}_
>     _{color:#000000}at org.apache.flink.util.Preconditions.checkArgument(Preconditions.java:138)
~[flink-dist_2.12-1.13.0.jar:1.13.0]{color}_
>     _{color:#000000}at org.apache.flink.runtime.rest.handler.job.JobExceptionsHandler.assertLocalExceptionInfo(JobExceptionsHandler.java:218)
~[flink-dist_2.12-1.13.0.jar:1.13.0]{color}_
>     _{color:#000000}at org.apache.flink.runtime.rest.handler.job.JobExceptionsHandler.createRootExceptionInfo(JobExceptionsHandler.java:191)
~[flink-dist_2.12-1.13.0.jar:1.13.0]{color}_
>     _{color:#000000}at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
~[?:?]{color}_
>     _{color:#000000}at java.util.stream.SliceOps$1$1.accept(SliceOps.java:199) ~[?:?]{color}_
>     _{color:#000000}at java.util.ArrayList$ArrayListSpliterator.tryAdvance(ArrayList.java:1632)
~[?:?]{color}_
>     _{color:#000000}at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:127)
~[?:?]{color}_
>     _{color:#000000}at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:502)
~[?:?]{color}_
>     _{color:#000000}at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:488)
~[?:?]{color}_
>     _{color:#000000}at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
~[?:?]{color}_
>     _{color:#000000}at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
~[?:?]{color}_
>     _{color:#000000}at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
~[?:?]{color}_
>     _{color:#000000}at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
~[?:?]{color}_
>     _{color:#000000}at org.apache.flink.runtime.rest.handler.job.JobExceptionsHandler.createJobExceptionHistory(JobExceptionsHandler.java:169)
~[flink-dist_2.12-1.13.0.jar:1.13.0]{color}_
>     _{color:#000000}at org.apache.flink.runtime.rest.handler.job.JobExceptionsHandler.createJobExceptionsInfo(JobExceptionsHandler.java:154)
~[flink-dist_2.12-1.13.0.jar:1.13.0]{color}_
>     _{color:#000000}at org.apache.flink.runtime.rest.handler.job.JobExceptionsHandler.handleRequest(JobExceptionsHandler.java:101)
~[flink-dist_2.12-1.13.0.jar:1.13.0]{color}_
>     _{color:#000000}at org.apache.flink.runtime.rest.handler.job.JobExceptionsHandler.handleRequest(JobExceptionsHandler.java:63)
~[flink-dist_2.12-1.13.0.jar:1.13.0]{color}_
>     _{color:#000000}at org.apache.flink.runtime.rest.handler.job.AbstractExecutionGraphHandler.lambda$handleRequest$0(AbstractExecutionGraphHandler.java:87)
~[flink-dist_2.12-1.13.0.jar:1.13.0]{color}_
>     _{color:#000000}at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:642)
[?:?]{color}_
>     _{color:#000000}at java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:478)
[?:?]{color}_
>     _{color:#000000}at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
[?:?]{color}_
>     _{color:#000000}at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]{color}_
>     _{color:#000000}at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
[?:?]{color}_
>     _{color:#000000}at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[?:?]{color}_
>     _{color:#000000}at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[?:?]{color}_
>     _{color:#000000}at java.lang.Thread.run(Thread.java:834) [?:?]{color}_
>  
> I see there are some exceptions in task managers and I remember the kind of exception
can be shown in UI in version 1.12.1 :
>  
> _2021-05-18 00:50:30,261 WARN org.apache.flink.runtime.taskmanager.Task [] - xxx (23/90)#13
(c345fb009b5d93628b5a6d890c8f4226) switched from RUNNING to FAILED with failure cause: org.apache.flink.runtime.io.network.netty.exception.RemoteTransportException:
Connection unexpectedly closed by remote task manager '10.194.65.3/10.194.65.3:44273'. This
might indicate that the remote task manager was lost._
>     _at org.apache.flink.runtime.io.network.netty.CreditBasedPartitionRequestClientHandler.channelInactive(CreditBasedPartitionRequestClientHandler.java:160)_
>     _at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)_
>     _at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)_
>     _at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:241)_
>     _at org.apache.flink.shaded.netty4.io.netty.channel.ChannelInboundHandlerAdapter.channelInactive(ChannelInboundHandlerAdapter.java:81)_
>     _at org.apache.flink.runtime.io.network.netty.NettyMessageClientDecoderDelegate.channelInactive(NettyMessageClientDecoderDelegate.java:94)_
>     _at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)_
>     _at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)_
>     _at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:241)_
>     _at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405)_
>     _at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)_
>     _at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)_
>     _at org.apache.flink.shaded.netty4.io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901)_
>     _at org.apache.flink.shaded.netty4.io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:818)_
>     _at org.apache.flink.shaded.netty4.io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)_
>     _at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)_
>     _at org.apache.flink.shaded.netty4.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)_
>     _at org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)_
>     _at org.apache.flink.shaded.netty4.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)_
>     _at java.base/java.lang.Thread.run(Thread.java:834)_
>  
>  
>  
> The issue has been reported in flink-user mailing list before: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Root-Exception-can-not-be-shown-on-Web-UI-in-Flink-1-13-0-td43673.html



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message