flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8818) Harden YarnFileStageTest upload test for eventual consistent read-after-write
Date Mon, 05 Mar 2018 17:30:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16386398#comment-16386398

ASF GitHub Bot commented on FLINK-8818:

Github user NicoK commented on the issue:

    The main issue was actually described in FLINK-8801:
    According to https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html#ConsistencyModel:
    > Amazon S3 provides read-after-write consistency for PUTS of new objects in your S3
bucket in all regions with one caveat. The caveat is that if you make a HEAD or GET request
to the key name (to find if the object exists) before creating the object, Amazon S3 provides
eventual consistency for read-after-write.
    Some S3 file system implementations may actually execute such a request for the about-to-write
object and thus the read-after-write is only eventually consistent.

> Harden YarnFileStageTest upload test for eventual consistent read-after-write
> -----------------------------------------------------------------------------
>                 Key: FLINK-8818
>                 URL: https://issues.apache.org/jira/browse/FLINK-8818
>             Project: Flink
>          Issue Type: Sub-task
>          Components: FileSystem, Tests, YARN
>    Affects Versions: 1.5.0
>            Reporter: Nico Kruber
>            Assignee: Nico Kruber
>            Priority: Blocker
>             Fix For: 1.5.0, 1.4.3

This message was sent by Atlassian JIRA

View raw message