flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-7582) Rest client may buffer responses indefinitely under heavy laod
Date Wed, 06 Sep 2017 13:39:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-7582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155342#comment-16155342
] 

ASF GitHub Bot commented on FLINK-7582:
---------------------------------------

Github user uce commented on a diff in the pull request:

    https://github.com/apache/flink/pull/4645#discussion_r137269070
  
    --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestClient.java ---
    @@ -159,24 +155,17 @@ public void shutdown(Time timeout) {
     	}
     
     	private <P extends ResponseBody> CompletableFuture<P> submitRequest(String
targetAddress, int targetPort, FullHttpRequest httpRequest, Class<P> responseClass)
{
    -		return CompletableFuture.supplyAsync(() -> bootstrap.connect(targetAddress, targetPort),
executor)
    -			.thenApply((channel) -> {
    -				try {
    -					return channel.sync();
    -				} catch (InterruptedException e) {
    -					throw new FlinkRuntimeException(e);
    -				}
    -			})
    -			.thenApply((ChannelFuture::channel))
    -			.thenCompose(channel -> {
    -				ClientHandler handler = channel.pipeline().get(ClientHandler.class);
    -				CompletableFuture<JsonResponse> future = handler.getJsonFuture();
    -				channel.writeAndFlush(httpRequest);
    -				return future;
    -			}).thenComposeAsync(
    -				(JsonResponse rawResponse) -> parseResponse(rawResponse, responseClass),
    -				executor
    -			);
    +		ChannelFuture connect = bootstrap.connect(targetAddress, targetPort);
    +		Channel channel;
    --- End diff --
    
    I understand, but it can be confusing that a method which returns a future actually has
a (potentially long) blocking operation.


> Rest client may buffer responses indefinitely under heavy laod
> --------------------------------------------------------------
>
>                 Key: FLINK-7582
>                 URL: https://issues.apache.org/jira/browse/FLINK-7582
>             Project: Flink
>          Issue Type: Bug
>          Components: REST
>    Affects Versions: 1.4.0
>            Reporter: Chesnay Schepler
>            Assignee: Chesnay Schepler
>             Fix For: 1.4.0
>
>
> The RestClient uses an executor for sending requests and parsing responses. Under heavy
load, i.e. lots of requests being sent, the executor may be used exclusively for sending requests.
The responses that are received by the netty threads are thus never parsed and are buffered
in memory, until either requests stop coming in or all memory is used up.
> We should let the netty receiver thread do the parsing as well.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message