sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Boglarka Egyed <b...@apache.org>
Subject Re: Review Request 68475: SQOOP-3363: Test incremental import with S3
Date Tue, 28 Aug 2018 12:04:54 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68475/
-----------------------------------------------------------

(Updated Aug. 28, 2018, 12:04 p.m.)


Review request for Sqoop, daniel voros, Fero Szabo, Nguyen Truong, and Szabolcs Vasas.


Changes
-------

Indentation fixed in S3TestUtils


Bugs: SQOOP-3363
    https://issues.apache.org/jira/browse/SQOOP-3363


Repository: sqoop-trunk


Description
-------

* Added new test cases for Parquet import into S3 as it was still missing
* Added new test cases for incremental append import into S3 in Text, Avro, Sequence and Parquet
file format
* Added new test cases for incremental merge import into S3 in Text and Parquet file format
* Updated some previously added logic in S3 util and test classes


Diffs (updated)
-----

  src/java/org/apache/sqoop/util/AppendUtils.java 20c0d13c391d312a943a147882cf85f86cb7e438

  src/java/org/apache/sqoop/util/FileSystemUtil.java 96ec21233d6378865195008f5ab67a74370fa2ed

  src/test/org/apache/sqoop/TestAppendUtils.java 3d66beca0226dd9eb8a14f4157444f24a247491a

  src/test/org/apache/sqoop/s3/TestS3AvroImport.java e130c42104b86e854d45babc009a5f1409a74a48

  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendAvroImport.java PRE-CREATION 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendParquetImport.java PRE-CREATION 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendSequenceFileImport.java PRE-CREATION

  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendTextImport.java PRE-CREATION 
  src/test/org/apache/sqoop/s3/TestS3IncrementalMergeParquetImport.java PRE-CREATION 
  src/test/org/apache/sqoop/s3/TestS3IncrementalMergeTextImport.java PRE-CREATION 
  src/test/org/apache/sqoop/s3/TestS3ParquetImport.java PRE-CREATION 
  src/test/org/apache/sqoop/s3/TestS3SequenceFileImport.java c17c1c54918df0b4d1ecbaef4e381975d72756ae

  src/test/org/apache/sqoop/s3/TestS3TextImport.java 60e2cd3025e67ecd43bdfb6b30d1b8d69a50da86

  src/test/org/apache/sqoop/testutil/AvroTestUtils.java 04a8494a5d1d8a5020d5a3b629bbab62d3c09ffd

  src/test/org/apache/sqoop/testutil/BaseSqoopTestCase.java ad2f10a071a0859c5b5e063c8cf5dda7c202124f

  src/test/org/apache/sqoop/testutil/S3TestUtils.java ceaff3b3a2bfd031b9772c9b43afdfa670c23718

  src/test/org/apache/sqoop/testutil/SequenceFileTestUtils.java ad7576dbb2447423c677429f24163031a9d39b5f

  src/test/org/apache/sqoop/testutil/TextFileTestUtils.java df19cb8be7a633a6f1e1e3f9bc7d0dbc268aa90a



Diff: https://reviews.apache.org/r/68475/diff/6/

Changes: https://reviews.apache.org/r/68475/diff/5-6/


Testing
-------

ant clean test -Ds3.bucket.url=<bucket-url> -Ds3.generator.command=<credential-generator-command>
./gradlew test -Ds3.bucket.url=<bucket-url> -Ds3.generator.command=<credential-generator-command>


Thanks,

Boglarka Egyed


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message