sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Boglarka Egyed (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SQOOP-3383) Disable fs.s3a.impl.disable.cache in S3 tests
Date Wed, 12 Sep 2018 06:38:00 GMT

     [ https://issues.apache.org/jira/browse/SQOOP-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Boglarka Egyed updated SQOOP-3383:
----------------------------------
    Description: 
FileSystem has a static cache meaning when the authentication happens in the org.apache.sqoop.testutil.S3TestUtils#setS3CredentialsInHadoopConf
method Sqoop import will get the same FileSystem object from the cache thus its authentication
via the \{{-Dfs.s3a.access.key}} and -{{Dfs.s3a.secret.key}} properties is effectless. See
org.apache.hadoop.fs.FileSystem#get(java.net.URI, org.apache.hadoop.conf.Configuration). 

This static cache should be disabled (by settin {{fs.s3a.impl.disable.cache}} to true) in
the setup phase of the S3 tests to make sure Sqoop relies on the S3 credentials set via the
-D properties.

  was:
FileSystem has a static cache meaning when the authentication happens in the org.apache.sqoop.testutil.S3TestUtils#setS3CredentialsInHadoopConf
method Sqoop import will get the same FileSystem object from the cache thus its authentication
via the -D{{fs.s3a.access.key}} and -{{Dfs.s3a.secret.key}} properties is effectless. See
org.apache.hadoop.fs.FileSystem#get(java.net.URI, org.apache.hadoop.conf.Configuration). 

This static cache should be disabled (by settin {{fs.s3a.impl.disable.cache}} to true) in
the setup phase of the S3 tests to make sure Sqoop relies on the S3 credentials set via the
-D properties.


> Disable fs.s3a.impl.disable.cache in S3 tests
> ---------------------------------------------
>
>                 Key: SQOOP-3383
>                 URL: https://issues.apache.org/jira/browse/SQOOP-3383
>             Project: Sqoop
>          Issue Type: Sub-task
>    Affects Versions: 1.4.7
>            Reporter: Boglarka Egyed
>            Assignee: Boglarka Egyed
>            Priority: Minor
>         Attachments: SQOOP-3383.patch
>
>
> FileSystem has a static cache meaning when the authentication happens in the org.apache.sqoop.testutil.S3TestUtils#setS3CredentialsInHadoopConf
method Sqoop import will get the same FileSystem object from the cache thus its authentication
via the \{{-Dfs.s3a.access.key}} and -{{Dfs.s3a.secret.key}} properties is effectless. See
org.apache.hadoop.fs.FileSystem#get(java.net.URI, org.apache.hadoop.conf.Configuration). 
> This static cache should be disabled (by settin {{fs.s3a.impl.disable.cache}} to true)
in the setup phase of the S3 tests to make sure Sqoop relies on the S3 credentials set via
the -D properties.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message