flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4174) Enhance Window Evictor
Date Tue, 01 Nov 2016 03:43:58 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15624238#comment-15624238
] 

ASF GitHub Bot commented on FLINK-4174:
---------------------------------------

GitHub user soniclavier opened a pull request:

    https://github.com/apache/flink/pull/2736

    [FLINK-4174] Enhance evictor functionality

    The PR implements [FLINK-4174](https://issues.apache.org/jira/browse/FLINK-4174) Enhance
window evictor as proposed in [FLIP-4](https://cwiki.apache.org/confluence/display/FLINK/FLIP-4+%3A+Enhance+Window+Evictor)
    
    Changes made:
    
    - Modified the Evictor API, added two new methods evictBefore and evictAfter. Removed
existing evict method
    - Modified the corresponding implementations of CountEvictor, DeltaEvictor and TimeEvictor
    - Created EvictorContext in the class EvictingWindowOperator, which is passed to the evictor
methods
    - Created TimestampedValue class which holds the value with corresponding timestamp. This
class is exposed to the user via the evictBefore and evictAfter methods.
    - Modified EvictingWindowOperator class 
    	- to call evictBefore before calling window function and evictAfter after calling window
function.
    	- create FluentIterable<TimestampedValue<IN>> from Iterable<StreamRecord<IN>>
contents, which is passed to the evictor methods.
    	- to clear the windowstate and add the remaining element(after the eviction) back to
the state. (this fixes [FLINK-4369](https://issues.apache.org/jira/browse/FLINK-4369))
    - Added test cases in EvictingWindowOperatorTest.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/soniclavier/flink FLINK-4174

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/2736.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2736
    
----
commit bfd4fb509463dea0dc86e702c3aad0b0b9e70ff2
Author: Vishnu Viswanath <vishnu.viswanath25@gmail.com>
Date:   2016-10-31T23:21:04Z

    [FLINK-4174] Enhance evictor functionality

----


> Enhance Window Evictor
> ----------------------
>
>                 Key: FLINK-4174
>                 URL: https://issues.apache.org/jira/browse/FLINK-4174
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Streaming
>            Reporter: vishnu viswanath
>            Assignee: vishnu viswanath
>
> Enhance the current functionality of Evictor as per this [design document|https://docs.google.com/document/d/1Rr7xzlItYqvFXLyyy-Yv0vvw8f29QYAjm5i9E4A_JlU/edit].
> This includes:
> - Allow eviction of elements from the window in any order (not only from the beginning).
To do this Evictor must go through the list of elements and remove the elements that have
to be evicted instead of the current approach of : returning the count of elements to be removed
from beginning.
> - Allow eviction to be done before/after applying the window function.
> FLIP page for this enhancement : [FLIP-4|https://cwiki.apache.org/confluence/display/FLINK/FLIP-4+%3A+Enhance+Window+Evictor]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message