beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kenneth Knowles (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-1656) DirectRunner should not call finalize twice in UnboundedSourceExecutorFactory
Date Thu, 09 Mar 2017 04:19:38 GMT

    [ https://issues.apache.org/jira/browse/BEAM-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15902452#comment-15902452
] 

Kenneth Knowles commented on BEAM-1656:
---------------------------------------

Haven't dropped into the code. But since the direct runner doesn't durably store anything,
it can basically call it whenever it want. Might be good to do so. Runners that do durably
store might generally use their in-memory version (without deserialization) to finalize.

> DirectRunner should not call finalize twice in UnboundedSourceExecutorFactory
> -----------------------------------------------------------------------------
>
>                 Key: BEAM-1656
>                 URL: https://issues.apache.org/jira/browse/BEAM-1656
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-direct
>            Reporter: Ben Chambers
>            Assignee: Thomas Groh
>
> In [getReader](https://github.com/apache/beam/blob/master/runners/direct-java/src/main/java/org/apache/beam/runners/direct/UnboundedReadEvaluatorFactory.java#L174)
we call finalize on the checkpoint that arrived in the incoming shard. Then later in [finishRead](https://github.com/apache/beam/blob/master/runners/direct-java/src/main/java/org/apache/beam/runners/direct/UnboundedReadEvaluatorFactory.java#L205)
we finalize the old checkpoint again.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message