flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gyula Fora (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-2923) Make it possible to mix-and-match StateBackends with KvState implementations
Date Mon, 26 Oct 2015 18:36:27 GMT
Gyula Fora created FLINK-2923:

             Summary: Make it possible to mix-and-match StateBackends with KvState implementations
                 Key: FLINK-2923
                 URL: https://issues.apache.org/jira/browse/FLINK-2923
             Project: Flink
          Issue Type: Improvement
          Components: Streaming
            Reporter: Gyula Fora
            Assignee: Gyula Fora

The KvState implementations are currently very closely tied to the specific StateBackend implementations
and one has to reimplement the Backend to change the KvState. For many applications it is
probably necessary to store the non-partitioned states differently from the key-value states.

An example would be a sql KvState with a simple file/memory backend for non-partitioned states
(like bloomfilters, and offsets).

A wrapper object should be provided to allow the use of a backend with a custom KvState. 

My proposal:
Create a KvStateProvider class which will have methods for initializing, creating and closing
KvStates (independent from the backend)

Create a wrapper StateBackend that wraps a StateBackend and a KvStateprovider into a new StateBackend.
This could be a static method of the StateBackend class:

`public static Statebackend StateBackend.createWithCustomKvState(StateBackend, KvStateProvider)`

This message was sent by Atlassian JIRA

View raw message