flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fabian Hueske (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8578) Implement rowtime DataStream to Table upsert conversion.
Date Mon, 15 Oct 2018 16:48:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16650471#comment-16650471
] 

Fabian Hueske commented on FLINK-8578:
--------------------------------------

bq. If a user wants to upsert under proct-time, he doesn't need to define a rowtime attribute
field in a table schema. The rowtime attribute will be converted to a regular TIMESTAMP attribute
after the upsert conversion.
bq. 

The problem is that rowtime attributes are always read from the internal StreamRecord timestamp
field. So unless, there is a copy of the field in the data type of the DataStream, a user
would not have access to that. But I agree, that seems to a not very common special case.

> Implement rowtime DataStream to Table upsert conversion.
> --------------------------------------------------------
>
>                 Key: FLINK-8578
>                 URL: https://issues.apache.org/jira/browse/FLINK-8578
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Table API &amp; SQL
>            Reporter: Hequn Cheng
>            Assignee: Hequn Cheng
>            Priority: Major
>
> Flink-8577 implements upsert from stream under proctime. This task is going to solve
the order problem introduce by proctime. As proposed by Fabian in FLINK-8545, it would be
good to be able to declare a time attribute that decides whether an upsert is performed or
not.
> {code:java}
> Table table = tEnv.upsertFromStream(input, 'a, 'b.rowtime.upsertOrder, 'c.key)
> {code}
> This is a good way to solve the order problem using rowtime. And an idea comes to my
mind that we can even remove the `.upsertOrder`, because the rowtime attribute can only be
defined once in a table schema. Removing `.upsertOrder` also makes it easier to design api
for TableSource and sql, i.e, we don't need to add another new feature for the api.
> Any suggestions are welcomed!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message