beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Kirpichov (JIRA)" <j...@apache.org>
Subject [jira] [Created] (BEAM-988) Support for testing how soon output is emitted
Date Tue, 15 Nov 2016 23:41:59 GMT
Eugene Kirpichov created BEAM-988:
-------------------------------------

             Summary: Support for testing how soon output is emitted
                 Key: BEAM-988
                 URL: https://issues.apache.org/jira/browse/BEAM-988
             Project: Beam
          Issue Type: Bug
            Reporter: Eugene Kirpichov
            Assignee: Thomas Groh


I ran into this issue when testing Splittable DoFn. My intention is, it should behave exactly
like a DoFn - i.e. emit output immediately when it receives input, regardless of the windowing/triggering
strategy of the input (even though SDF has a GBK internally).

However, currently the SDK doesn't have facilities for testing that. TestStream allows controlling
the timing of the input, but there's nothing to capture timing of the output. Moreover, timing
of the output is unspecified by the model because triggers technically only enable firing,
but do not force it (i.e. they are a lower bound on when output will be emitted).

I'm not sure what's the best way to address this. E.g., perhaps, PaneInfo could include a
field "since when was this pane enabled to fire" (regardless of when it really fired)?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message