flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-7310) always use HybridMemorySegment
Date Tue, 01 Aug 2017 13:14:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16108870#comment-16108870
] 

ASF GitHub Bot commented on FLINK-7310:
---------------------------------------

Github user StephanEwen commented on the issue:

    https://github.com/apache/flink/pull/4445
  
    These changes look good to me!
    
    There is in fact a potential performance impact of this change. It would be cool to get
an understanding of the potential performance impact of only using the HybridMemorySegment
now.
    We could run something like a Hash Join Performance test with key/value pairs of String
keys (which are the most performance sensitive to serialize / deserialize with individual
byte operations) and see if this has a measurable impact there.


> always use HybridMemorySegment
> ------------------------------
>
>                 Key: FLINK-7310
>                 URL: https://issues.apache.org/jira/browse/FLINK-7310
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Core
>    Affects Versions: 1.4.0
>            Reporter: Nico Kruber
>            Assignee: Nico Kruber
>
> For future changes to the network buffers (sending our own off-heap buffers through to
netty), we cannot use {{HeapMemorySegment}} anymore and need to rely on {{HybridMemorySegment}}
instead.
> We should thus drop any code that loads the {{HeapMemorySegment}} (it is still available
if needed) in favour of the {{HybridMemorySegment}} which is able to work on both heap and
off-heap memory.
> FYI: For the performance penalty of this change compared to using {{HeapMemorySegment}}
alone, see this interesting blob article (from 2015):
> https://flink.apache.org/news/2015/09/16/off-heap-memory.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message