hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (Jira)" <j...@apache.org>
Subject [jira] [Work logged] (HADOOP-17590) ABFS: Introduce Lease Operations with Append to provide single writer semantics
Date Wed, 02 Jun 2021 10:49:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-17590?focusedWorklogId=605101&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-605101
]

ASF GitHub Bot logged work on HADOOP-17590:
-------------------------------------------

                Author: ASF GitHub Bot
            Created on: 02/Jun/21 10:48
            Start Date: 02/Jun/21 10:48
    Worklog Time Spent: 10m 
      Work Description: snvijaya commented on a change in pull request #3026:
URL: https://github.com/apache/hadoop/pull/3026#discussion_r643841953



##########
File path: hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java
##########
@@ -521,12 +523,14 @@ public OutputStream createFile(final Path path,
             isNamespaceEnabled ? getOctalNotation(permission) : null,
             isNamespaceEnabled ? getOctalNotation(umask) : null,
             isAppendBlob,
-            null);
+            null,
+            lease);
       }
       perfInfo.registerResult(op.getResult()).registerSuccess(true);
 
-      AbfsLease lease = maybeCreateLease(relativePath);
-
+      if (lease == null) {

Review comment:
       Acquiring lease along with create/append enables atomic fetch of lease with the store
operation and prevent any parallel writers. If configured for infinite lease, the path creation
will happen without bundled lease acquiry and rely on later acquireLease API call in AbfsLease.

   AbfsLease background acquireLease should be scheduled post lease is acquired along with
create API (bundled) ? Infinite lease flow purely on acquireLease API should be the case only
if fs.azure.write.enforceLease is off.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 605101)
    Time Spent: 1h 40m  (was: 1.5h)

> ABFS: Introduce Lease Operations with Append to provide single writer semantics
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-17590
>                 URL: https://issues.apache.org/jira/browse/HADOOP-17590
>             Project: Hadoop Common
>          Issue Type: Sub-task
>            Reporter: Sneha Varma
>            Assignee: Sneha Varma
>            Priority: Major
>              Labels: pull-request-available
>          Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> The lease operations will be introduced as part of Append, Flush to ensure the single
writer semantics.
>  
> Details:
> Acquire Lease will be introduced in Create, Auto-Renew, Acquire will be added to Append
& Release, Auto-Renew, Acquire in Flush.
>  
> Duration the creation of the file the lease will be acquired, as part of appends the
lease will be auto-renewed & the lease can be released as part of flush.
>  
> By default the lease duration will be of 60 seconds.
> "fs.azure.write.enforcelease" & "fs.azure.write.lease.duration" two configs will
be introduced.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message