flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4705) Instrument FixedLengthRecordSorter
Date Mon, 10 Oct 2016 23:52:21 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15563930#comment-15563930

ASF GitHub Bot commented on FLINK-4705:

GitHub user greghogan opened a pull request:


    [FLINK-4705] Instrument FixedLengthRecordSorter

    Updates comparators with support for key normalization.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/greghogan/flink 4705_instrument_fixedlengthrecordsorter

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2617
commit 10d0955251ad9965da2094682ea1d4c468602373
Author: Greg Hogan <code@greghogan.com>
Date:   2016-09-28T16:55:03Z

    [FLINK-4705] Instrument FixedLengthRecordSorter
    Updates comparators with support for key normalization.


> Instrument FixedLengthRecordSorter
> ----------------------------------
>                 Key: FLINK-4705
>                 URL: https://issues.apache.org/jira/browse/FLINK-4705
>             Project: Flink
>          Issue Type: Improvement
>          Components: Local Runtime
>    Affects Versions: 1.2.0
>            Reporter: Greg Hogan
>            Assignee: Greg Hogan
> The {{NormalizedKeySorter}} sorts on the concatenation of (potentially partial) keys
plus an 8-byte pointer to the record. After sorting each pointer must be dereferenced, which
is not cache friendly.
> The {{FixedLengthRecordSorter}} sorts on the concatentation of full keys followed by
the remainder of the record. The records can then be deserialized in sequence.
> Instrumenting the {{FixedLengthRecordSorter}} requires implementing the comparator methods
{{writereadWithKeyNormalization}} and {{readWithKeyNormalization}}.
> Testing {{JaccardIndex}} on an m4.16xlarge the scale 18 runtime dropped from 71.8 to
68.8 s (4.3% faster) and the scale 20 runtime dropped from 546.1 to 501.8 s (8.8% faster).

This message was sent by Atlassian JIRA

View raw message