sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Boglarka Egyed <b...@apache.org>
Subject Re: Review Request 68677: SQOOP-3383: Eliminate duplicated steps in S3 tests
Date Wed, 12 Sep 2018 06:39:00 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68677/
-----------------------------------------------------------

(Updated Sept. 12, 2018, 6:38 a.m.)


Review request for Sqoop, daniel voros, Fero Szabo, and Szabolcs Vasas.


Changes
-------

Updated description


Bugs: SQOOP-3383
    https://issues.apache.org/jira/browse/SQOOP-3383


Repository: sqoop-trunk


Description (updated)
-------

FileSystem has a static cache meaning when the authentication happens in the org.apache.sqoop.testutil.S3TestUtils#setS3CredentialsInHadoopConf
method Sqoop import will get the same FileSystem object from the cache thus its authentication
via the -Dfs.s3a.access.key and -Dfs.s3a.secret.key properties is effectless. See org.apache.hadoop.fs.FileSystem#get(java.net.URI,
org.apache.hadoop.conf.Configuration). 

This static cache should be disabled (by settin fs.s3a.impl.disable.cache to true) in the
setup phase of the S3 tests to make sure Sqoop relies on the S3 credentials set via the -D
properties.


Diffs
-----

  src/test/org/apache/sqoop/testutil/S3TestUtils.java 7724026b0bad25a31aa76c89135a51538b46bf82



Diff: https://reviews.apache.org/r/68677/diff/2/


Testing
-------

./gradlew test -Ds3.bucket.url=<bucket-url> -Ds3.generator.command=<credential-generator-command>


Thanks,

Boglarka Egyed


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message