flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "vinoyang (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (FLINK-9278) Allow restore savepoint with some SQL queries added/removed
Date Fri, 22 Mar 2019 11:41:00 GMT

     [ https://issues.apache.org/jira/browse/FLINK-9278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

vinoyang reassigned FLINK-9278:

    Assignee: vinoyang

> Allow restore savepoint with some SQL queries added/removed
> -----------------------------------------------------------
>                 Key: FLINK-9278
>                 URL: https://issues.apache.org/jira/browse/FLINK-9278
>             Project: Flink
>          Issue Type: Improvement
>          Components: Table SQL / API
>    Affects Versions: 1.4.2
>            Reporter: Adrian Hains
>            Assignee: vinoyang
>            Priority: Major
> We are running a Flink job that contains multiple SQL queries. This is configured by
calling sqlQuery(String) one time for each SQL query, on a single instance of StreamTableEnvironment.
The queries are simple aggregations with a tumble window.
> Currently I can configure my environment with queries Q1, Q2, and Q3, create a savepoint,
and restart the job from that savepoint if the same set of SQL queries are used.
> If I remove some queries and add some others, Q2, Q4, and Q3, I am unable to restart
the job from the same savepoint. This behavior is expected, as the documentation clearly describes
that the operator IDs are generated if they are not explicitly defined, and they cannot be
explicitly defined when using flink SQL.
> I would like to be able to specify a scoping operator id prefix when registering a SQL
query to a StreamTableEnvironment. This can then be used to programmatically generate unique
IDs for each of the operators created to execute the SQL queries. For example, if I specify
a prefix of "ID:Q2:" for my Q2 query, and I restart the job with an identical SQL query for
this prefix, then I would be able to restore the state for this query even in the presence
of other queries being added or removed to the job graph.

This message was sent by Atlassian JIRA

View raw message