flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-5615) queryable state: execute the QueryableStateITCase for all three state back-ends
Date Mon, 23 Jan 2017 15:27:26 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15834739#comment-15834739
] 

ASF GitHub Bot commented on FLINK-5615:
---------------------------------------

GitHub user NicoK opened a pull request:

    https://github.com/apache/flink/pull/3194

    [FLINK-5615][query] execute the QueryableStateITCase for all three state back-ends

    This extends the `QueryableStateITCase` so that it is able to run with any selected state
backend. Some optimisations reduce the total runtime of the test cases so that we are able
to run the tests with all three currently available backends, i.e. `MemoryStateBackend`, `FsStateBackend`,
and `RocksDBStateBackend`, with little extra costs.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/NicoK/flink flink-5615

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/3194.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3194
    
----
commit 9c1a247ada0015ff1b97c6017e8c1de874ba6d17
Author: Nico Kruber <nico@data-artisans.com>
Date:   2017-01-17T13:26:16Z

    [FLINK-5613][query] querying a non-existing key is inconsistent among state backends
    
    Querying for a non-existing key for a state that has a default value set
    currently results in an UnknownKeyOrNamespace exception when the
    MemoryStateBackend or FsStateBackend is used but results in the default value
    if RocksDBStateBackend is set.
    
    This removes the special handling from the RocksDBStateBackend and makes it
    consistent with the other two back-ends, i.e. returning null which results
    in the mentioned UnknownKeyOrNamespace exception.

commit cccbc5f30e09f5ccbc61a75fc9519dacf91c5522
Author: Nico Kruber <nico@data-artisans.com>
Date:   2017-01-23T14:17:46Z

    [FLINK-5615][query] improve testQueryableStateWithTaskManagerFailure test duration
    
    This is based on the following points:
    * slow down QueryableStateITCase#TestKeyRangeSource for the rest of the
      program to make more progress (we do not need a full-speed source here!)
    * reduce the checkpointing interval
    * reduce the amount of progress before starting our own evaluation
    * reduce the number of checkpoints to wait for before killing a TM
    * reduce the thread waiting time when asking how many checkpoints exist
    
    Note that by slowing down QueryableStateITCase#TestKeyRangeSource, the other
    tests should only be affected positively, too, since they also did not really
    need a full-speed source and thus have more CPU cycles for their own tasks.
    
    This, among with

commit 3dd506dd65f6f9f9e8879e6bf6df0261435d5317
Author: Nico Kruber <nico@data-artisans.com>
Date:   2017-01-23T14:47:40Z

    [FLINK-5615][query] speed up some more tests in QueryableStateITCase
    
    This is based on reducing the number of keys the source generates. We do not
    really need 1024 different keys for the tests - go with 256 now.

commit ee6b78eb3a91f39386148e3ef1c55a0f3824843f
Author: Nico Kruber <nico@data-artisans.com>
Date:   2017-01-23T14:57:09Z

    [FLINK-5615][query] execute the QueryableStateITCase for all state back-ends

----


> queryable state: execute the QueryableStateITCase for all three state back-ends
> -------------------------------------------------------------------------------
>
>                 Key: FLINK-5615
>                 URL: https://issues.apache.org/jira/browse/FLINK-5615
>             Project: Flink
>          Issue Type: Improvement
>          Components: Queryable State
>    Affects Versions: 1.2.0
>            Reporter: Nico Kruber
>            Assignee: Nico Kruber
>
> The QueryableStateITCase currently is only tested with the MemoryStateBackend but as
has been seen in the past, some errors or inconsistent behaviour only appeared with different
state back-ends. It should thus be extended to be tested with all three currently existing
state back-ends.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message