flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4942) Improve processing performance of HeapInternalTimerService with key groups
Date Fri, 28 Oct 2016 14:53:59 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15615623#comment-15615623
] 

ASF GitHub Bot commented on FLINK-4942:
---------------------------------------

Github user kl0u commented on the issue:

    https://github.com/apache/flink/pull/2702
  
    Are we sure that this is going to lead to improvements? I understand that both cases have
pros and cons, as in one case we pay the price at each arriving timer, while in the other
you pay it at once upon checkpointing. Actually, there was the same discussion when designing
how to make the timers rescalable and we ended up with the existing version.  


> Improve processing performance of HeapInternalTimerService with key groups
> --------------------------------------------------------------------------
>
>                 Key: FLINK-4942
>                 URL: https://issues.apache.org/jira/browse/FLINK-4942
>             Project: Flink
>          Issue Type: Improvement
>            Reporter: Stefan Richter
>            Assignee: Stefan Richter
>
> Currently, key groups awareness in `HeapInternalTimerService` is basically implemented
as (hash) map of (hash) sets. Purpose of this is grouping key groups together in a way that
allows easy serialization into key groups.
> However, this data layout comes along with some significant performance decrease, in
particular when the number of key groups is high.
> I suggest to keep all timers in one set again at runtime, thus being as fast as in previous
versions without key groups.
> Instead, we can perform a very fast online partitioning into key groups before a snapshot.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message