flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Metzger (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2732) Add access to the TaskManagers' log file and out file in the web dashboard.
Date Thu, 15 Oct 2015 10:01:05 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14958601#comment-14958601
] 

Robert Metzger commented on FLINK-2732:
---------------------------------------

"The idea is not to have any communication between task managers" ? To implement this feature,
there is no communication required between TMs.

I don't think that the three network hops [~sachingoel0101] is mentioning are a big issue.
There are a lot of messages send back and forth between JM and TM all the time.
Some more arguments against Sachin's proposal:
- Running a web server on each TaskManager just for serving some log files also requires resources
- We can not assume that users can access the TaskManagers from their browsers. Remember that
in many companies, there are firewalls in place, which allow only access to certain nodes.
The web interface of Flink on YARN is also accessed through a proxy running on the resource
manager. The RM will then load the resources from the AM/JM container. So we can not assume
that we can establish a connection from client browser <---> TM container on YARN.
That's also the reason why we send the data of the {{DataSet.collect()}} through the JM.

> Add access to the TaskManagers' log file and out file in the web dashboard.
> ---------------------------------------------------------------------------
>
>                 Key: FLINK-2732
>                 URL: https://issues.apache.org/jira/browse/FLINK-2732
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Webfrontend
>    Affects Versions: 0.10
>            Reporter: Stephan Ewen
>            Assignee: Martin Liesenberg
>             Fix For: 0.10
>
>
> Add access to the TaskManagers' log file and out file in the web dashboard.
> This needs addition on the server side, as the log files need to be transferred   to
the JobManager via the blob server.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message