flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Juho Autio <juho.au...@rovio.com>
Subject REST API "broken" on YARN because POST is not allowed via YARN proxy
Date Wed, 04 Apr 2018 13:02:53 GMT
I just learned that Flink savepoints API was refactored to require using
HTTP POST.

That's fine otherwise, but makes life harder when Flink is run on top of
YARN.

I've added example calls below to show how POST is declined by
the hadoop-yarn-server-web-proxy*, which only supports GET and PUT.

Can you think of any solution to this? If I would be able to determine the
actual host & port for Flink UI, I could use that instead of the proxy
address. But that would probably require opening at least one more port, so
it's not the optimal solution either. Ideally I would have Flink REST API
completely accessible with GET and PUT methods.

To me it seems like AWS EMR will also hit this issue as soon as they start
supporting Flink 1.5, because they seem to run Flink as a YARN app.


*)
https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/WebAppProxyServlet.java#L296-L306


$ http POST
http://10.0.10.71:20888/proxy/application_1522844153347_0001/jobs/652e207f8578574d4a322e23d4f8b908/checkpoints

HTTP/1.1 405 HTTP method POST is not supported by this URL
Cache-Control: must-revalidate,no-cache,no-store
Content-Length: 1523
Content-Type: text/html; charset=iso-8859-1
Date: Wed, 04 Apr 2018 12:48:05 GMT
Date: Wed, 04 Apr 2018 12:48:05 GMT
Pragma: no-cache
Pragma: no-cache

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
<title>Error 405 HTTP method POST is not supported by this URL</title>
</head>
<body><h2>HTTP ERROR 405</h2>
<p>Problem accessing
/proxy/application_1522844153347_0001/jobs/652e207f8578574d4a322e23d4f8b908/checkpoints.
Reason:
<pre>    HTTP method POST is not supported by this URL</pre></p><hr
/><i><small>Powered by Jetty://</small></i><br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>
<br/>

</body>
</html>


$ http PUT
http://10.0.10.71:20888/proxy/application_1522844153347_0001/jobs/652e207f8578574d4a322e23d4f8b908/checkpoints

HTTP/1.1 404 Not Found
Access-Control-Allow-Origin: *
Cache-Control: no-cache
Content-Length: 25
Content-Type: application/json; charset=UTF-8
Date: Wed, 04 Apr 2018 12:48:09 GMT
Date: Wed, 04 Apr 2018 12:48:09 GMT
Expires: Wed, 04 Apr 2018 12:48:09 GMT
Expires: Wed, 04 Apr 2018 12:48:09 GMT
Pragma: no-cache
Pragma: no-cache

{
    "errors": [
        "Not found."
    ]
}

^ expected to get this from flink because there's no PUT /checkpoints.

Mime
View raw message