flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [flink] GJL commented on a change in pull request #10152: [FLINK-14466][runtime] Let YarnJobClusterEntrypoint use user code class loader
Date Thu, 14 Nov 2019 17:09:48 GMT
GJL commented on a change in pull request #10152: [FLINK-14466][runtime] Let YarnJobClusterEntrypoint
use user code class loader
URL: https://github.com/apache/flink/pull/10152#discussion_r346438265
 
 

 ##########
 File path: flink-yarn/src/test/java/org/apache/flink/yarn/YarnFileStageTest.java
 ##########
 @@ -126,7 +130,18 @@ public void testCopyFromLocalRecursiveWithoutScheme() throws Exception
{
 		final FileSystem targetFileSystem = hdfsRootPath.getFileSystem(hadoopConfig);
 		final Path targetDir = targetFileSystem.getWorkingDirectory();
 
-		testCopyFromLocalRecursive(targetFileSystem, targetDir, tempFolder, false);
+		testCopyFromLocalRecursive(targetFileSystem, targetDir, LOCAL_RESOURCE_DIRECTORY, tempFolder,
false);
+	}
+
+	/**
+	 * Verifies that a single file is properly copied.
+	 */
+	@Test
+	public void testCopyAFileFromLocal() throws IOException, URISyntaxException, InterruptedException
{
 
 Review comment:
   Alright, I can see now there is a different branch. However, there is quite some redundancy
between `testCopyAFileFromLocal` and `testCopyFromLocalRecursive`. Unfortunately, I cannot
give you a solution on the spot. BTW, your test will pass even if we simplify the code in
`uploadAndRegisterFiles` to
   
   ```
   	for (File shipFile : shipFiles) {
   			// add directories to the classpath
   			final java.nio.file.Path shipPath = shipFile.toPath();
   			final java.nio.file.Path parentPath = shipPath.getParent();
   			Files.walkFileTree(shipPath, new SimpleFileVisitor<java.nio.file.Path>() {
   				@Override
   				public FileVisitResult visitFile(java.nio.file.Path file, BasicFileAttributes attrs)
{
   					localPaths.add(new Path(file.toUri()));
   					relativePaths.add(new Path(localResourcesDirectory, parentPath.relativize(file).toString()));
   					return FileVisitResult.CONTINUE;
   				}
   			});
   	}
   ``` 
   
   The reason is that `Files.walkFileTree` also works if the first argument is not a directory.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message