flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sunjincheng (Jira)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-16497) Improve default flush strategy for JDBC sink to make it work out-of-box
Date Mon, 01 Jun 2020 10:57:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-16497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120927#comment-17120927

sunjincheng commented on FLINK-16497:

Hi [~libenchao] , Thanks for your reply!

I think is is difficult for us to set an optimal default value from the perspective of performance,
which is related to specific business and storage. So at the design level, for flow computing
scenarios, real-time insertion (1 row) is a better semantic expression. So I think about it
in terms of semantics and real-time. Even though the time of 1s is very short, beginners will
still feel that it is real-time calculation, but not real-time write to storage, but mini-batch
(1s may have multiple records), So, for now, I still prefer 1 row as default value. :)

What do you think?

> Improve default flush strategy for JDBC sink to make it work out-of-box
> -----------------------------------------------------------------------
>                 Key: FLINK-16497
>                 URL: https://issues.apache.org/jira/browse/FLINK-16497
>             Project: Flink
>          Issue Type: Improvement
>          Components: Connectors / JDBC, Table SQL / Ecosystem
>            Reporter: Jark Wu
>            Priority: Major
>             Fix For: 1.11.0
> Currently, JDBC sink provides 2 flush options:
> {code}
> 'connector.write.flush.max-rows' = '5000', -- default is 5000
> 'connector.write.flush.interval' = '2s', -- no default value
> {code}
> That means if flush interval is not set, the buffered output rows may not be flushed
to database for a long time. That is a surprising behavior because no results are outputed
by default. 
> So I propose to have a default flush '1s' interval for JDBC sink or default 1 row for
flush size. 

This message was sent by Atlassian Jira

View raw message