flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jark Wu (Jira)" <j...@apache.org>
Subject [jira] [Closed] (FLINK-17887) Improve interface of ScanFormatFactory and SinkFormatFactory
Date Mon, 01 Jun 2020 02:39:00 GMT

     [ https://issues.apache.org/jira/browse/FLINK-17887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jark Wu closed FLINK-17887.
---------------------------
    Resolution: Fixed

Fixed in
 - master (1.12.0): 64de78e36500b5a8c8720639ded4d1c5f963ad41
 - 1.11.0: 5b98a9ee338c01bfe276464aee0080b72ca1bf6c

> Improve interface of ScanFormatFactory and SinkFormatFactory
> ------------------------------------------------------------
>
>                 Key: FLINK-17887
>                 URL: https://issues.apache.org/jira/browse/FLINK-17887
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Table SQL / API
>            Reporter: Jark Wu
>            Assignee: Jark Wu
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 1.11.0
>
>
> There is some problem with current ScanForamtFactory and SinkFormatFactory interfaces:
> 1) {{ScanFormat#createScanFormat}} only accepts {{ScanTableSource.Context}}, which means
it can’t work in lookup source.
> 2) The naming of {{ScanFormat}} also indicates it is only used in scan source. But a
lookup source should be able to work with format too.
> 3) It’s confusing that {{ScanFormatFactory#createScanFormat}} and {{ScanFormat#createScanFormat}}
(create itself?)
> The proposed new interface strucutre:
> {code:java}
> interface DeserializationFormatFactory {
>   DeserializationSchemaProvider createDeserializationSchemaProvider(
>       DynamicTableFactory.Context context, ReadableConfig formatOptions);
> }
> interface SerializationFormatFactory {
>   SerializationSchemaProvider createSerializationSchemaProvider(
>        DynamicTableFactory.Context context, ReadableConfig formatOptions);
> }
> interface DeserializationSchemaProvider {
>   DeserializationSchema createDeserializationSchema(
>       DynamicTableSource.Context context, DataType producedDataType)
> }
> interface SerializationSchemaProvider{
>   SerializationSchema createSerializationSchema(
>       DynamicTableSink.Context context, DataType consumedDataType); 
> }
> {code}
> The top-level user-facing interfaces are still {{DeserializationFormatFactory}} and {{SerializationFormatFactory}}.
This is also good for future evolution, e.g. introducing new {{EncoderFormatFactory}} and
{{DecoderFormatFactory}} if we have a new runtime {{Encoder}} {{Decoder}} in the future. This
also solves our concerns: no concepts of SourceFormat and SinkFormat .



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message