hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [hadoop] bharatviswa504 commented on a change in pull request #1631: HDDS-2188 : Implement LocatedFileStatus & getFileBlockLocations to pr…
Date Wed, 09 Oct 2019 23:04:45 GMT
bharatviswa504 commented on a change in pull request #1631: HDDS-2188 : Implement LocatedFileStatus
& getFileBlockLocations to pr…
URL: https://github.com/apache/hadoop/pull/1631#discussion_r333270214
 
 

 ##########
 File path: hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java
 ##########
 @@ -688,6 +663,48 @@ public OmKeyInfo lookupKey(OmKeyArgs args, String clientAddress)
     }
   }
 
+  /**
+   * Refresh pipeline info in OM by asking SCM.
+   * @param value OmKeyInfo
+   */
+  @VisibleForTesting
+  protected void refreshPipeline(OmKeyInfo value) {
+    // Refresh container pipeline info from SCM
+    // based on OmKeyArgs.refreshPipeline flag
+    // 1. Client send initial read request OmKeyArgs.refreshPipeline = false
+    // and uses the pipeline cached in OM to access datanode
+    // 2. If succeeded, done.
+    // 3. If failed due to pipeline does not exist or invalid pipeline state
+    //    exception, client should retry lookupKey with
+    //    OmKeyArgs.refreshPipeline = true
+    Map<Long, ContainerWithPipeline> containerWithPipelineMap = new HashMap<>();
+    for (OmKeyLocationInfoGroup key : value.getKeyLocationVersions()) {
+      key.getLocationList().forEach(k -> {
+        // TODO: fix Some tests that may not initialize container client
+        // The production should always have containerClient initialized.
+        if (scmClient.getContainerClient() != null) {
+          ContainerWithPipeline cp = containerWithPipelineMap
+              .computeIfAbsent(k.getContainerID(), cId -> {
+                try {
+                  return scmClient.getContainerClient()
+                      .getContainerWithPipeline(cId);
+                } catch (IOException e) {
+                  LOG.error("Unable to update pipeline for container:{}",
+                      k.getContainerID());
+                  return null;
 
 Review comment:
   Do we we need to fail here, instead of logging as error and return null. As when reading
this will eventually fail right, as no pipeline info is available for that container.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message