flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-7310) always use HybridMemorySegment
Date Wed, 06 Sep 2017 02:21:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16154711#comment-16154711
] 

ASF GitHub Bot commented on FLINK-7310:
---------------------------------------

Github user KurtYoung commented on the issue:

    https://github.com/apache/flink/pull/4445
  
    I would bet on deserialization for it. And why sorter suffers more regression than hash
join is that sorter will cause more deserializations during compare records than hash join.
    
    Despite the regression we will face, i think it's still worthy since we can avoid an extra
copy from network to runtime. It's better if we can take the extra copy into account during
benchmark, but it's ok we don't have it. 
    
    +1 to merge this.


> always use HybridMemorySegment
> ------------------------------
>
>                 Key: FLINK-7310
>                 URL: https://issues.apache.org/jira/browse/FLINK-7310
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Core
>    Affects Versions: 1.4.0
>            Reporter: Nico Kruber
>            Assignee: Nico Kruber
>
> For future changes to the network buffers (sending our own off-heap buffers through to
netty), we cannot use {{HeapMemorySegment}} anymore and need to rely on {{HybridMemorySegment}}
instead.
> We should thus drop any code that loads the {{HeapMemorySegment}} (it is still available
if needed) in favour of the {{HybridMemorySegment}} which is able to work on both heap and
off-heap memory.
> FYI: For the performance penalty of this change compared to using {{HeapMemorySegment}}
alone, see this interesting blob article (from 2015):
> https://flink.apache.org/news/2015/09/16/off-heap-memory.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message