flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [flink] Myasuka commented on a change in pull request #7942: [FLINK-11696][checkpoint] Avoid to send mkdir requests to DFS from task side
Date Mon, 11 Mar 2019 16:52:09 GMT
Myasuka commented on a change in pull request #7942: [FLINK-11696][checkpoint] Avoid to send
mkdir requests to DFS from task side
URL: https://github.com/apache/flink/pull/7942#discussion_r264328294
 
 

 ##########
 File path: flink-runtime/src/main/java/org/apache/flink/runtime/state/filesystem/FsCheckpointStorage.java
 ##########
 @@ -107,6 +105,13 @@ public CheckpointStorageLocation initializeLocationForCheckpoint(long
checkpoint
 		// prepare all the paths needed for the checkpoints
 		final Path checkpointDir = createCheckpointDirectory(checkpointsDirectory, checkpointId);
 
+		if (!areDirectoriesCreated) {
+			fileSystem.mkdirs(checkpointsDirectory);
+			fileSystem.mkdirs(sharedStateDirectory);
 
 Review comment:
   At first, I planed to add a new initialization method to create those directories. However,
I found current `initializeLocationForCheckpoint` method already has some initialization semantics.
I think since all checkpoints would first start from checkpoint coordinator, and no clear
annotation for `CheckpointStorage` I'm not sure adding a new method would whether break the
compatibility.
   
   For open-close-principle, I think it would be better to just adding an initialization method.
And to be honest, I would not find any benefits in lazy initialization.
   
   If adding a new method, do you have any suggestion about the method name to distinguish
from current `initializeLocationForCheckpoint`?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message