beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Work logged] (BEAM-5250) Python Wordcount fails with Flink portable streaming
Date Tue, 04 Sep 2018 10:16:01 GMT

     [ https://issues.apache.org/jira/browse/BEAM-5250?focusedWorklogId=140832&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-140832
]

ASF GitHub Bot logged work on BEAM-5250:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 04/Sep/18 10:15
            Start Date: 04/Sep/18 10:15
    Worklog Time Spent: 10m 
      Work Description: mxm commented on issue #6315: [BEAM-5250] Fix GlobalWindow's max timestamp
in Python SDK
URL: https://github.com/apache/beam/pull/6315#issuecomment-418314834
 
 
   >Having the end of the GlobalWindow be max - 1 day seems rather arbitrary. 
   
   Yes, it is arbitrary. The reason is stated in `GlobalWindow` (just restating in case somebody
didn't have the chance to look it up):
   ```
     // Triggers use maxTimestamp to set timers' timestamp. Timers fires when
     // the watermark passes their timestamps. So, the maxTimestamp needs to be
     // smaller than the TIMESTAMP_MAX_VALUE.
     // One standard day is subtracted from TIMESTAMP_MAX_VALUE to make sure
     // the maxTimestamp is smaller than TIMESTAMP_MAX_VALUE even after rounding up
     // to seconds or minutes.
   ```
   
   >I think it'd be worth doing at least a cursory exploration into what it would take
to "fix" Java. If that's too hard, at least we should make this a proto value (in standard
windows perhaps)?
   
   I suppose it would work to trigger all windows when the Watermark is at MAX_TIMESTAMP.
However, that is a change in semantics, since the max window timestamp normally needs to be
larger than the current Watermark. Since we can't make any further progress, that could be
ok.
   
   Otherwise, implementing the GlobalWindow timestamp in the RunnerAPI would be the most portable
approach. 
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 140832)
    Time Spent: 1h 10m  (was: 1h)

> Python Wordcount fails with Flink portable streaming
> ----------------------------------------------------
>
>                 Key: BEAM-5250
>                 URL: https://issues.apache.org/jira/browse/BEAM-5250
>             Project: Beam
>          Issue Type: Improvement
>          Components: runner-flink
>            Reporter: Thomas Weise
>            Assignee: Maximilian Michels
>            Priority: Major
>              Labels: portability
>          Time Spent: 1h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message