beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt Pouttu-Clarke (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-91) Retractions
Date Thu, 23 Jun 2016 00:48:16 GMT

    [ https://issues.apache.org/jira/browse/BEAM-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345507#comment-15345507
] 

Matt Pouttu-Clarke commented on BEAM-91:
----------------------------------------

With regards versioning structural changes, removing a field is one example as is changing
the type of a field.  In this case one must replay all relevant history with the change applied
AND more importantly quickly identify the root cause of failures related to the structural
change. 

With regard to retaining "deleted" data and relationships, the best real example I have are
versioned hierarchical structures like zip codes and sales territories.  You cannot reject
mail because the zip code has changed or moved, and sales people will have a conniption if
their numbers change and effect their commissions.  Thus in the real world these historical
structures remain frozen in time potentially forever even when they are "deleted".



> Retractions
> -----------
>
>                 Key: BEAM-91
>                 URL: https://issues.apache.org/jira/browse/BEAM-91
>             Project: Beam
>          Issue Type: New Feature
>          Components: beam-model
>            Reporter: Tyler Akidau
>            Assignee: Frances Perry
>   Original Estimate: 672h
>  Remaining Estimate: 672h
>
> We still haven't added retractions to Beam, even though they're a core part of the model.
We should document all the necessary aspects (uncombine, reverting DoFn output with DoOvers,
sink integration, source-level retractions, etc), and then implement them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message