flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Till Rohrmann (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-12075) Not able to submit jobs on YARN when there's a firewall
Date Mon, 01 Apr 2019 19:25:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-12075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16807100#comment-16807100
] 

Till Rohrmann commented on FLINK-12075:
---------------------------------------

I agree that we should fix this problem for the {{1.8.0}} release. I suggest the following
as a solution: 

1. only set the Yarn bind-port to {{0}} if it has not been specified ({{rest.bind-port}} and
all of its fallback keys have not been set).
2. Remove the explicit setting of {{rest.port}} from the default config

That way, it should be possible to define a port range for {{rest.bind-port}} and keep the
behaviour that you only need to specify a single option {{rest.port}} if you want to bind
and connect to a single port which existing setups might depend on. Moreover, with 2. the
Yarn case should work out of the box with random port binding.

> Not able to submit jobs on YARN when there's a firewall
> -------------------------------------------------------
>
>                 Key: FLINK-12075
>                 URL: https://issues.apache.org/jira/browse/FLINK-12075
>             Project: Flink
>          Issue Type: Bug
>          Components: Deployment / YARN
>    Affects Versions: 1.7.2, 1.8.0
>            Reporter: Richard Deurwaarder
>            Priority: Blocker
>             Fix For: 1.8.0
>
>
> If there is a firewall around the YARN cluster and the machine, submitting flink job
it is unpractical because new flink clusters start up with random ports for REST communication.
>  
> FLINK-5758 should've fixed this. But it seems FLINK-11081 either undid the changes or
did not implement this. The relevant code is changed in FLINK-11081 ([https://github.com/apache/flink/commit/730eed71ef3f718d61f85d5e94b1060844ca56db#diff-487838863ab693af7008f04cb3359be3R102])
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message