hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Fabbri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14946) S3Guard testPruneCommandCLI can fail
Date Thu, 24 May 2018 06:30:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16488508#comment-16488508

Aaron Fabbri commented on HADOOP-14946:

I hope you don't mind [~gabor.bota], I've attached a v2 patch that is stable for me even with
extreme load on my system.  Differences from v1:
 * Add a big comment to {{testPruneCommand()}} explaining what it is doing.
 * If time between creation of "fresh" file and completion of prune command is > max path
age, just warn() and skip the problematic assertion. I can only reproduce this if I torture
my system with 18 test threads and some unrelated benchmarks running to slow down my system.
 * Use a constant.

[~gabor.bota] if you like this patch I suggest we commit this and keep an eye out for the
other case (Steve's most recent stack trace). I do not have more time to debug it now, but
I would probably re-review the prune CLI handling of "-seconds 1" and if that looks good,
maybe look at [~stevel@apache.org]'s NTP logs for big adjustments during his test run?? 
Other ideas?

> S3Guard testPruneCommandCLI can fail
> ------------------------------------
>                 Key: HADOOP-14946
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14946
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.0.0
>            Reporter: Steve Loughran
>            Assignee: Gabor Bota
>            Priority: Major
>         Attachments: HADOOP-14946.001.patch, HADOOP-14946.002.patch
> The test of the S3Guard CLI prune can sometimes fail on parallel test runs. Assumption:
it is the parallelism which is causing the problem
> {code}
> org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB
> testPruneCommandCLI(org.apache.hadoop.fs.s3a.s3guard.ITestS3GuardToolDynamoDB)  Time
elapsed: 10.765 sec  <<< FAILURE!
> java.lang.AssertionError: Pruned children count [] expected:<1> but was:<0>
> 	at org.junit.Assert.fail(Assert.java:88)
> {code}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message