sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Szabolcs Vasas <vasas.szabo...@gmail.com>
Subject Re: Review Request 68677: SQOOP-3383: Disable FileSystem static cache in S3 tests
Date Wed, 12 Sep 2018 07:01:29 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68677/#review208549
-----------------------------------------------------------


Ship it!




Thank you for submitting the fix, let's ship it!

- Szabolcs Vasas


On Sept. 12, 2018, 6:42 a.m., Boglarka Egyed wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68677/
> -----------------------------------------------------------
> 
> (Updated Sept. 12, 2018, 6:42 a.m.)
> 
> 
> Review request for Sqoop, daniel voros, Fero Szabo, and Szabolcs Vasas.
> 
> 
> Bugs: SQOOP-3383
>     https://issues.apache.org/jira/browse/SQOOP-3383
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> FileSystem has a static cache meaning when the authentication happens in the org.apache.sqoop.testutil.S3TestUtils#setS3CredentialsInHadoopConf
method Sqoop import will get the same FileSystem object from the cache thus its authentication
via the -Dfs.s3a.access.key and -Dfs.s3a.secret.key properties is effectless. See org.apache.hadoop.fs.FileSystem#get(java.net.URI,
org.apache.hadoop.conf.Configuration). 
> 
> This static cache should be disabled (by settin fs.s3a.impl.disable.cache to true) in
the setup phase of the S3 tests to make sure Sqoop relies on the S3 credentials set via the
-D properties.
> 
> 
> Diffs
> -----
> 
>   src/test/org/apache/sqoop/testutil/S3TestUtils.java 7724026b0bad25a31aa76c89135a51538b46bf82

> 
> 
> Diff: https://reviews.apache.org/r/68677/diff/2/
> 
> 
> Testing
> -------
> 
> ./gradlew test -Ds3.bucket.url=<bucket-url> -Ds3.generator.command=<credential-generator-command>
> 
> 
> Thanks,
> 
> Boglarka Egyed
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message