flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [flink] StefanRRichter commented on a change in pull request #7942: [FLINK-11696][checkpoint] Avoid to send mkdir requests to DFS from task side
Date Mon, 11 Mar 2019 17:03:56 GMT
StefanRRichter commented on a change in pull request #7942: [FLINK-11696][checkpoint] Avoid
to send mkdir requests to DFS from task side
URL: https://github.com/apache/flink/pull/7942#discussion_r264333733
 
 

 ##########
 File path: flink-runtime/src/main/java/org/apache/flink/runtime/state/filesystem/FsCheckpointStorage.java
 ##########
 @@ -107,6 +105,13 @@ public CheckpointStorageLocation initializeLocationForCheckpoint(long
checkpoint
 		// prepare all the paths needed for the checkpoints
 		final Path checkpointDir = createCheckpointDirectory(checkpointsDirectory, checkpointId);
 
+		if (!areDirectoriesCreated) {
+			fileSystem.mkdirs(checkpointsDirectory);
+			fileSystem.mkdirs(sharedStateDirectory);
 
 Review comment:
   I think it would be reasonable to add a new method. As for the names, maybe something like
a `initializeLocation` and `initializeLocationForCheckpoint` -- meaning (as we can explain
in the docs):  one method initializes that initializes the location _in general_ and one method
that initializes things for a specific checkpoint x. So shared directories etc can go to general
init, and the checkpoint specific folder goes to the existing method. 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message