flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8818) Harden YarnFileStageTest upload test for eventual consistent read-after-write
Date Mon, 05 Mar 2018 17:30:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16386398#comment-16386398
] 

ASF GitHub Bot commented on FLINK-8818:
---------------------------------------

Github user NicoK commented on the issue:

    https://github.com/apache/flink/pull/5601
  
    The main issue was actually described in FLINK-8801:
    
    According to https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html#ConsistencyModel:
    
    > Amazon S3 provides read-after-write consistency for PUTS of new objects in your S3
bucket in all regions with one caveat. The caveat is that if you make a HEAD or GET request
to the key name (to find if the object exists) before creating the object, Amazon S3 provides
eventual consistency for read-after-write.
    
    
    Some S3 file system implementations may actually execute such a request for the about-to-write
object and thus the read-after-write is only eventually consistent.


> Harden YarnFileStageTest upload test for eventual consistent read-after-write
> -----------------------------------------------------------------------------
>
>                 Key: FLINK-8818
>                 URL: https://issues.apache.org/jira/browse/FLINK-8818
>             Project: Flink
>          Issue Type: Sub-task
>          Components: FileSystem, Tests, YARN
>    Affects Versions: 1.5.0
>            Reporter: Nico Kruber
>            Assignee: Nico Kruber
>            Priority: Blocker
>             Fix For: 1.5.0, 1.4.3
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message