beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Work logged] (BEAM-4176) Java: Portable batch runner passes all ValidatesRunner tests that non-portable runner passes
Date Tue, 18 Sep 2018 22:10:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-4176?focusedWorklogId=145504&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-145504
]

ASF GitHub Bot logged work on BEAM-4176:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 18/Sep/18 22:09
            Start Date: 18/Sep/18 22:09
    Worklog Time Spent: 10m 
      Work Description: ryan-williams commented on issue #6328: [BEAM-4176] enumerate primitive
transforms in portable construction
URL: https://github.com/apache/beam/pull/6328#issuecomment-422573991
 
 
   OK, I managed to get [a run to complete](https://storage.googleapis.com/runsascoded-tmp/beam-portable-flink-vr-tests/ecd1ac085a/index.html):
   
   ![](https://cl.ly/4ed744829264/Screen%20Shot%202018-09-18%20at%206.04.22%20PM.png)
   
   Notes on getting it to run:
   - I increased my local Docker image size to 256GB (from 64GB)
   - I also ran `docker container prune -f` multiple times during the run
   - idk if either of the above are necessary or sufficient
   - `docker system df` is also useful to determine how full the image is.
   
   Notes on the results:
   - 10 more cases are passing since I started this PR
     - possibly due to fixes I added for unittests over the course of this PR
     - or other OOB changes
   - runtime is much worse! 2h30m. seems like nothing to be done there atm, and is related
to an increased number of containers being created (thanks for explaining @angoenka)
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 145504)
    Time Spent: 21h 40m  (was: 21.5h)

> Java: Portable batch runner passes all ValidatesRunner tests that non-portable runner
passes
> --------------------------------------------------------------------------------------------
>
>                 Key: BEAM-4176
>                 URL: https://issues.apache.org/jira/browse/BEAM-4176
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-flink
>            Reporter: Ben Sidhom
>            Priority: Major
>         Attachments: Screen Shot 2018-08-14 at 4.18.31 PM.png, Screen Shot 2018-09-03
at 11.07.38 AM.png
>
>          Time Spent: 21h 40m
>  Remaining Estimate: 0h
>
> We need this as a sanity check that runner execution is correct.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message