hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [hadoop] sidseth commented on a change in pull request #1115: HADOOP-16207 testMR failures
Date Tue, 01 Oct 2019 16:53:47 GMT
sidseth commented on a change in pull request #1115: HADOOP-16207 testMR failures
URL: https://github.com/apache/hadoop/pull/1115#discussion_r330161806
 
 

 ##########
 File path: hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/s3a/commit/AbstractYarnClusterITest.java
 ##########
 @@ -61,45 +69,70 @@
  * If two subclasses of this class are instantiated in the same JVM, in order,
  * they are guaranteed to be isolated.
  *
- * History: this is a superclass extracted from
- * {@link AbstractITCommitMRJob} while adding support for testing terasorting.
- *
  */
 public abstract class AbstractYarnClusterITest extends AbstractCommitITest {
 
   private static final Logger LOG =
       LoggerFactory.getLogger(AbstractYarnClusterITest.class);
 
-  private static final int TEST_FILE_COUNT = 2;
-  private static final int SCALE_TEST_FILE_COUNT = 50;
+  private static final int TEST_FILE_COUNT = 1;
+  private static final int SCALE_TEST_FILE_COUNT = 2;
 
-  public static final int SCALE_TEST_KEYS = 1000;
+  public static final int SCALE_TEST_KEYS = 100;
   public static final int BASE_TEST_KEYS = 10;
 
   private boolean scaleTest;
 
-  private boolean uniqueFilenames = false;
+  /**
+   * The static cluster binding with the lifecycle of this test; served
+   * through instance-level methods for sharing across methods in the
+   * suite.
+   */
+  @SuppressWarnings("StaticNonFinalField")
+  private static ClusterBinding clusterBinding;
 
 Review comment:
   I'm not sure how well this is going to work, given there's at least 2 tests which are inheriting
from this class.
   
   Either could end up creating the cluster, and then terminating the cluster in '@AfterClass'
without knowing what the other test is doing - and I think this will lead to failures depending
on the timing of executing ITestTerasortOnS3A and ITestS3ACommitterMRJob.
   
   These tests likely need there own clusters, and teardown methods. Or maybe the parameter
can be made non-static, so that individual test classes don't mess with each other. (They
get their own clusters as a result). Afaik, the individual test classes will still be able
to share clusters.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message