hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-16229) wasb LocalSASKeyGeneratorImpl only supports HTTP connections
Date Tue, 02 Apr 2019 11:58:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-16229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16807687#comment-16807687
] 

Steve Loughran commented on HADOOP-16229:
-----------------------------------------

 debug log (with a couple of extra statements)
{code}
 bin/hadoop fs -touch  wasbs://store1@stevelabfsinterop.blob.core.windows.net/file1
2019-04-02 12:56:43,763 [main] DEBUG azure.AzureNativeFileSystemStore (AzureNativeFileSystemStore.java:initialize(535))
- Secure mode enabled, local SAS flag = true
2019-04-02 12:56:43,767 [main] DEBUG azure.SecureStorageInterfaceImpl (SecureStorageInterfaceImpl.java:<init>(86))
- Authenticating with SecureStorage and local SAS key
2019-04-02 12:56:43,773 [main] DEBUG azure.CachingAuthorizer (CachingAuthorizer.java:init(68))
- SASKEY : Initializing CachingAuthorizer instance
2019-04-02 12:56:43,774 [main] DEBUG azure.SecureStorageInterfaceImpl (SecureStorageInterfaceImpl.java:<init>(101))
- Container SAS key will be used for all access
2019-04-02 12:56:43,777 [main] DEBUG azure.AzureNativeFileSystemStore (AzureNativeFileSystemStore.java:configureAzureStorageSession(823))
- AzureNativeFileSystemStore init. Settings=8,false,90,{3000,3000,30000,30},{true,1.0,1.0}
2019-04-02 12:56:43,778 [main] DEBUG azure.AzureNativeFileSystemStore (AzureNativeFileSystemStore.java:connectToAzureStorageInSecureMode(916))
- Connecting to Azure storage in Secure Mode
2019-04-02 12:56:43,792 [main] DEBUG azure.LocalSASKeyGeneratorImpl (LocalSASKeyGeneratorImpl.java:getContainerSASUri(83))
- Retrieving Container SAS URI For store1@stevelabfsinterop.blob.core.windows.net
2019-04-02 12:56:43,793 [main] DEBUG azure.CachingAuthorizer (CachingAuthorizer.java:get(92))
- SASKEY: CACHE MISS: stevelabfsinterop.blob.core.windows.net:store1:/
2019-04-02 12:56:43,793 [main] DEBUG azure.LocalSASKeyGeneratorImpl (LocalSASKeyGeneratorImpl.java:getSASKeyBasedStorageAccountInstance(124))
- Creating SAS key from account instance stevelabfsinterop.blob.core.windows.net
2019-04-02 12:56:43,943 [main] DEBUG azure.LocalSASKeyGeneratorImpl (LocalSASKeyGeneratorImpl.java:getContainerSASUri(96))
- Created SAS key for URI http://stevelabfsinterop.blob.core.windows.net/store1
2019-04-02 12:56:43,945 [main] DEBUG azure.LocalSASKeyGeneratorImpl (LocalSASKeyGeneratorImpl.java:getContainerSASUri(98))
- Transfomred URI is http://stevelabfsinterop.blob.core.windows.net/store1?ss=b&sig=*****************D&api-version=2017-07-29&se=2019-07-01T11%3A56%3A43Z&sv=2017-07-29&srt=co&sp=racwdlu&sr=co
2019-04-02 12:56:43,945 [main] DEBUG azure.CachingAuthorizer (CachingAuthorizer.java:put(102))
- SASKEY: CACHE PUT: stevelabfsinterop.blob.core.windows.net:store1:/, http://stevelabfsinterop.blob.core.windows.net/store1?ss=b&sig==*****************D&api-version=2017-07-29&se=2019-07-01T11%3A56%3A43Z&sv=2017-07-29&srt=co&sp=racwdlu&sr=co
2019-04-02 12:56:43,950 [main] DEBUG azure.AzureNativeFileSystemStore (AzureNativeFileSystemStore.java:initialize(550))
- Page blob directories:  
2019-04-02 12:56:43,950 [main] DEBUG azure.AzureNativeFileSystemStore (AzureNativeFileSystemStore.java:initialize(558))
- Block blobs with compaction directories:  
2019-04-02 12:56:43,950 [main] DEBUG azure.AzureNativeFileSystemStore (AzureNativeFileSystemStore.java:initialize(575))
- Atomic rename directories: /hbase 
2019-04-02 12:56:43,950 [main] DEBUG azure.NativeAzureFileSystem (NativeAzureFileSystem.java:initialize(1352))
- NativeAzureFileSystem. Initializing.
2019-04-02 12:56:43,952 [main] DEBUG azure.NativeAzureFileSystem (NativeAzureFileSystem.java:initialize(1353))
-   blockSize  = 536870912
2019-04-02 12:56:43,997 [main] DEBUG azure.NativeAzureFileSystem (NativeAzureFileSystem.java:getFileStatus(2604))
- Getting the file status for wasbs://store1@stevelabfsinterop.blob.core.windows.net/file1
2019-04-02 12:56:43,998 [main] DEBUG azure.AzureNativeFileSystemStore (AzureNativeFileSystemStore.java:retrieveMetadata(2126))
- Retrieving metadata for file1
2019-04-02 12:56:44,038 [main] DEBUG azure.SelfThrottlingIntercept (SelfThrottlingIntercept.java:sendingRequest(167))
-  SelfThrottlingIntercept:: SendingRequest:   threadId=1, requestType=read , isFirstRequest=true,
sleepDuration=0
2019-04-02 12:56:44,142 [main] DEBUG azure.SelfThrottlingIntercept (SelfThrottlingIntercept.java:responseReceived(115))
- SelfThrottlingIntercept:: ResponseReceived: threadId=1, Status=400, Elapsed(ms)=103, ETAG=null,
contentLength=-1, requestMethod=HEAD
touch: com.microsoft.azure.storage.StorageException: The account being accessed does not support
http.
2019-04-02 12:56:44,149 [shutdown-hook-0] DEBUG azure.NativeAzureFileSystem (NativeAzureFileSystem.java:close(3582))
- Submitting metrics when file system closed took 0 ms.
{code}


> wasb LocalSASKeyGeneratorImpl only supports HTTP connections
> ------------------------------------------------------------
>
>                 Key: HADOOP-16229
>                 URL: https://issues.apache.org/jira/browse/HADOOP-16229
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs/azure
>    Affects Versions: 3.2.0, 3.1.2
>            Reporter: Steve Loughran
>            Priority: Minor
>
> If you enable local sas and secure mode, you can't interact with https-only stores.
> This seems to be because the client created in {{LocalSASKeyGeneratorImpl}} doesn't pass
in the URL.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message