flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [flink] GJL commented on a change in pull request #10152: [FLINK-14466][runtime] Let YarnJobClusterEntrypoint use user code class loader
Date Thu, 14 Nov 2019 17:09:30 GMT
GJL commented on a change in pull request #10152: [FLINK-14466][runtime] Let YarnJobClusterEntrypoint
use user code class loader
URL: https://github.com/apache/flink/pull/10152#discussion_r346438265

 File path: flink-yarn/src/test/java/org/apache/flink/yarn/YarnFileStageTest.java
 @@ -126,7 +130,18 @@ public void testCopyFromLocalRecursiveWithoutScheme() throws Exception
 		final FileSystem targetFileSystem = hdfsRootPath.getFileSystem(hadoopConfig);
 		final Path targetDir = targetFileSystem.getWorkingDirectory();
-		testCopyFromLocalRecursive(targetFileSystem, targetDir, tempFolder, false);
+		testCopyFromLocalRecursive(targetFileSystem, targetDir, LOCAL_RESOURCE_DIRECTORY, tempFolder,
+	}
+	/**
+	 * Verifies that a single file is properly copied.
+	 */
+	@Test
+	public void testCopyAFileFromLocal() throws IOException, URISyntaxException, InterruptedException
 Review comment:
   Alright, I can see now there is a different branch. However, there is quite some redundancy
between `testCopyAFileFromLocal` and `testCopyFromLocalRecursive`. Unfortunately, I cannot
give you a solution on the spot. BTW, your test would will pass even if we simplify the code
in `uploadAndRegisterFiles` to
   	for (File shipFile : shipFiles) {
   			// add directories to the classpath
   			final java.nio.file.Path shipPath = shipFile.toPath();
   			final java.nio.file.Path parentPath = shipPath.getParent();
   			Files.walkFileTree(shipPath, new SimpleFileVisitor<java.nio.file.Path>() {
   				public FileVisitResult visitFile(java.nio.file.Path file, BasicFileAttributes attrs)
   					localPaths.add(new Path(file.toUri()));
   					relativePaths.add(new Path(localResourcesDirectory, parentPath.relativize(file).toString()));
   					return FileVisitResult.CONTINUE;
   The reason is that `Files.walkFileTree` also works if the first argument is not a directory.

This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:

With regards,
Apache Git Services

View raw message