flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-7582) Rest client may buffer responses indefinitely under heavy laod
Date Tue, 05 Sep 2017 11:02:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-7582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16153474#comment-16153474

ASF GitHub Bot commented on FLINK-7582:

GitHub user zentol opened a pull request:


    [FLINK-7582] [REST] Netty thread immediately parses responses

    ## What is the purpose of the change
    This PR modifies the RestClient to make the netty receiver threads parse responses (and
complete the associated future) immediately after receiving a response, instead of deferring
the parsing to a separate executor.
    This guarantees that we don't buffer responses that haven't been parsed yet. This was
previously the case if the executor could not keep up with the response rate, for example
because it was used exclusively for sending requests.
    ## Verifying this change
    This change is a trivial rework / code cleanup without any test coverage.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/zentol/flink 7582

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #4645
commit bf910ff69fa2754c235f8f0d957cadefbc37c53f
Author: zentol <chesnay@apache.org>
Date:   2017-09-05T10:58:57Z

    [FLINK-7582] [REST] Netty thread immediately parses responses


> Rest client may buffer responses indefinitely under heavy laod
> --------------------------------------------------------------
>                 Key: FLINK-7582
>                 URL: https://issues.apache.org/jira/browse/FLINK-7582
>             Project: Flink
>          Issue Type: Bug
>          Components: REST
>    Affects Versions: 1.4.0
>            Reporter: Chesnay Schepler
>            Assignee: Chesnay Schepler
>             Fix For: 1.4.0
> The RestClient uses an executor for sending requests and parsing responses. Under heavy
load, i.e. lots of requests being sent, the executor may be used exclusively for sending requests.
The responses that are received by the netty threads are thus never parsed and are buffered
in memory, until either requests stop coming in or all memory is used up.
> We should let the netty receiver thread do the parsing as well.

This message was sent by Atlassian JIRA

View raw message