flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8818) Harden YarnFileStageTest upload test for eventual consistent read-after-write
Date Mon, 05 Mar 2018 16:53:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16386341#comment-16386341
] 

ASF GitHub Bot commented on FLINK-8818:
---------------------------------------

Github user StephanEwen commented on the issue:

    https://github.com/apache/flink/pull/5601
  
    S3 is actually strongly consistent when reading newly created objects, just not in listing
or renaming objects (files).
    
    The test seems to actually use reads of full paths, so wondering why there is a failure
in the first place.
    
    If there is an issue that the Yarn upload code relies on eventually consistent operations,
then fixing the test by retries may disguise the actual issue. If there is no eventually consistent
operation, then this should not be necessary in the first place. I fear this change may be
down a tricky path...
    
    Can you explain/double check why the failure happened and why the retry is necessary to
stabilize the test, but the actual Yarn code is not affected by this?


> Harden YarnFileStageTest upload test for eventual consistent read-after-write
> -----------------------------------------------------------------------------
>
>                 Key: FLINK-8818
>                 URL: https://issues.apache.org/jira/browse/FLINK-8818
>             Project: Flink
>          Issue Type: Sub-task
>          Components: FileSystem, Tests, YARN
>    Affects Versions: 1.5.0
>            Reporter: Nico Kruber
>            Assignee: Nico Kruber
>            Priority: Blocker
>             Fix For: 1.5.0, 1.4.3
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message