hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-16644) Intermittent failure of ITestS3ATerasortOnS3A: timestamp differences
Date Tue, 08 Oct 2019 12:18:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-16644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16946787#comment-16946787
] 

Steve Loughran commented on HADOOP-16644:
-----------------------------------------

We really need a way of getting that FS timestamp off the store. I am "reluctant" to do it
in a HEAD straight after the create, but it is the only way to guarantee consistency. Doing
the head/update during the PUT would also address HADOOP-16412 (etag and version) and keep
[~sseth] happy.

+![~gabor.bota], [~fabbri]

*we could always think about making that HEAD/PUT async, though that could lead to even more
inconsistency pain.



> Intermittent failure of ITestS3ATerasortOnS3A: timestamp differences
> --------------------------------------------------------------------
>
>                 Key: HADOOP-16644
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16644
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3, test
>    Affects Versions: 3.3.0
>         Environment: -Dparallel-tests -DtestsThreadCount=8 -Dfailsafe.runOrder=balanced
-Ds3guard -Ddynamo -Dscale
> h2. Hypothesis:
> the timestamp of the source file is being picked up from S3Guard, but when the NM does
a getFileStatus call, a HEAD check is made -and this (due to the overloaded test system) is
out of sync with the listing. S3Guard is updated, the corrected date returned and the localisation
fails.
>            Reporter: Steve Loughran
>            Priority: Major
>
> Terasort of directory committer failing in resource localisaton -the partitions.lst file
has a different TS from that expected
> Happens under loaded integration tests (threads = 8; not standalone); non-auth s3guard
> {code}
> 2019-10-08 11:50:29,774 [IPC Server handler 4 on 55983] WARN  localizer.ResourceLocalizationService
(ResourceLocalizationService.java:processHeartbeat(1150)) - { s3a://hwdev-steve-ireland-new/terasort-directory/sortout/_partition.lst,
1570531828143, FILE, null } failed: Resource s3a://hwdev-steve-ireland-new/terasort-directory/sortout/_partition.lst
changed on src filesystem (expected 1570531828143, was 1570531828000
> java.io.IOException: Resource s3a://hwdev-steve-ireland-new/terasort-directory/sortout/_partition.lst
changed on src filesystem (expected 1570531828143, was 1570531828000
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message