beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Halperin (JIRA)" <>
Subject [jira] [Updated] (BEAM-146) WindowFn.AssignContext leaks implementation details about compressed WindowedValue representation
Date Sat, 25 Jun 2016 16:57:37 GMT


Daniel Halperin updated BEAM-146:
    Fix Version/s: 0.2.0-incubating

> WindowFn.AssignContext leaks implementation details about compressed WindowedValue representation
> -------------------------------------------------------------------------------------------------
>                 Key: BEAM-146
>                 URL:
>             Project: Beam
>          Issue Type: Bug
>          Components: beam-model
>            Reporter: Kenneth Knowles
>            Assignee: Kenneth Knowles
>            Priority: Minor
>             Fix For: 0.2.0-incubating
> Today, {{WindowFn.AssignContext}} provides simultaneous access to all of the windows
that a value has been placed in.
> Providing access to the current window for a value is convenient for, e.g. converting
day windows to hour windows for each hour of the assign day. But providing access to all the
assigned windows allows spooky action across windows, and is generally not intended to be
observable - elements are semantically considered to be "duplicated" into each of the assigned
> This ticket proposes that the {{AssignContext}} should provide only a single window,
and that windows should be "exploded" prior to window re-assignment so that elements are only
observed within one window at a time. This can be accomplished trivially today via surgical
insertion of {{RequiresWindowAccess}} but the {{AssignContext}} should have its API adjusted
to be explicit about it, too.
> This will affect only pipelines for which _all_ of the following hold:
>  - assigns to sliding windows (or custom {{WindowFn}} that places each element in multiple
>  - re-assigns to different windows without a {{GroupByKey}} between.
>  - the new window assignment actually does depend on the full set of windows assigned
> I hypothesize the number of such pipelines is zero.
> I expect to address this during the Beam Runner API design.

This message was sent by Atlassian JIRA

View raw message