flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-7582) Rest client may buffer responses indefinitely under heavy laod
Date Wed, 06 Sep 2017 09:01:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-7582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16155030#comment-16155030

ASF GitHub Bot commented on FLINK-7582:

Github user kl0u commented on a diff in the pull request:

    --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/rest/RestClient.java ---
    @@ -77,14 +77,10 @@
     	private static final ObjectMapper objectMapper = RestMapperUtils.getStrictObjectMapper();
    -	// used to open connections to a rest server endpoint
    -	private final Executor executor;
     	private Bootstrap bootstrap;
    -	public RestClient(RestClientConfiguration configuration, Executor executor) {
    +	public RestClient(RestClientConfiguration configuration) {
    --- End diff --
    Given that now we let netty do all the parsing, I think it makes sense to allow the user
to specify the size of the `NioEventLoopGroup ` (currently set to 1 - which can be the default).

> Rest client may buffer responses indefinitely under heavy laod
> --------------------------------------------------------------
>                 Key: FLINK-7582
>                 URL: https://issues.apache.org/jira/browse/FLINK-7582
>             Project: Flink
>          Issue Type: Bug
>          Components: REST
>    Affects Versions: 1.4.0
>            Reporter: Chesnay Schepler
>            Assignee: Chesnay Schepler
>             Fix For: 1.4.0
> The RestClient uses an executor for sending requests and parsing responses. Under heavy
load, i.e. lots of requests being sent, the executor may be used exclusively for sending requests.
The responses that are received by the netty threads are thus never parsed and are buffered
in memory, until either requests stop coming in or all memory is used up.
> We should let the netty receiver thread do the parsing as well.

This message was sent by Atlassian JIRA

View raw message