flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gary Yao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4534) Lack of synchronization in BucketingSink#restoreState()
Date Wed, 18 Jul 2018 15:01:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16547934#comment-16547934
] 

Gary Yao commented on FLINK-4534:
---------------------------------

[~yuzhihong@gmail.com] Synchronization bears a performance penalty. Synchronization makes
the code harder to read. We should avoid unnecessary synchronization by defining clear contracts
and threading models. Ideally, every line of code in the Flink repository should be idiomatic
because it possibly serves as a role model for future contributions.


> Lack of synchronization in BucketingSink#restoreState()
> -------------------------------------------------------
>
>                 Key: FLINK-4534
>                 URL: https://issues.apache.org/jira/browse/FLINK-4534
>             Project: Flink
>          Issue Type: Bug
>          Components: Streaming Connectors
>            Reporter: Ted Yu
>            Assignee: zhangminglei
>            Priority: Major
>
> Iteration over state.bucketStates is protected by synchronization in other methods, except
for the following in restoreState():
> {code}
>     for (BucketState<T> bucketState : state.bucketStates.values()) {
> {code}
> and following in close():
> {code}
>     for (Map.Entry<String, BucketState<T>> entry : state.bucketStates.entrySet())
{
>       closeCurrentPartFile(entry.getValue());
> {code}
> w.r.t. bucketState.pendingFilesPerCheckpoint , there is similar issue starting line 752:
> {code}
>       Set<Long> pastCheckpointIds = bucketState.pendingFilesPerCheckpoint.keySet();
>       LOG.debug("Moving pending files to final location on restore.");
>       for (Long pastCheckpointId : pastCheckpointIds) {
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message