sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Boglarka Egyed <b...@apache.org>
Subject Re: Review Request 68475: SQOOP-3363: Test incremental import with S3
Date Thu, 23 Aug 2018 18:18:38 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68475/
-----------------------------------------------------------

(Updated Aug. 23, 2018, 6:18 p.m.)


Review request for Sqoop, daniel voros, Fero Szabo, Nguyen Truong, and Szabolcs Vasas.


Changes
-------

Added temporary rootdir clean-up logic at the end of every test case


Bugs: SQOOP-3363
    https://issues.apache.org/jira/browse/SQOOP-3363


Repository: sqoop-trunk


Description
-------

* Added new test cases for Parquet import into S3 as it was still missing
* Added new test cases for incremental append import into S3 in Text, Avro, Sequence and Parquet
file format
* Added new test cases for incremental merge import into S3 in Text and Parquet file format
* Updated some previously added logic in S3 util and test classes


Diffs (updated)
-----

  src/test/org/apache/sqoop/s3/TestS3AvroImport.java e130c42104b86e854d45babc009a5f1409a74a48

  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendAvroImport.java PRE-CREATION 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendParquetImport.java PRE-CREATION 
  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendSequenceFileImport.java PRE-CREATION

  src/test/org/apache/sqoop/s3/TestS3IncrementalAppendTextImport.java PRE-CREATION 
  src/test/org/apache/sqoop/s3/TestS3IncrementalMergeParquetImport.java PRE-CREATION 
  src/test/org/apache/sqoop/s3/TestS3IncrementalMergeTextImport.java PRE-CREATION 
  src/test/org/apache/sqoop/s3/TestS3ParquetImport.java PRE-CREATION 
  src/test/org/apache/sqoop/s3/TestS3SequenceFileImport.java c17c1c54918df0b4d1ecbaef4e381975d72756ae

  src/test/org/apache/sqoop/s3/TestS3TextImport.java 60e2cd3025e67ecd43bdfb6b30d1b8d69a50da86

  src/test/org/apache/sqoop/testutil/AvroTestUtils.java 04a8494a5d1d8a5020d5a3b629bbab62d3c09ffd

  src/test/org/apache/sqoop/testutil/BaseSqoopTestCase.java 1730698e80cc77395f8a296b7bf01c104533e10b

  src/test/org/apache/sqoop/testutil/ParquetFileTestUtils.java PRE-CREATION 
  src/test/org/apache/sqoop/testutil/S3TestUtils.java ceaff3b3a2bfd031b9772c9b43afdfa670c23718

  src/test/org/apache/sqoop/testutil/SequenceFileTestUtils.java ad7576dbb2447423c677429f24163031a9d39b5f

  src/test/org/apache/sqoop/testutil/TextFileTestUtils.java df19cb8be7a633a6f1e1e3f9bc7d0dbc268aa90a



Diff: https://reviews.apache.org/r/68475/diff/3/

Changes: https://reviews.apache.org/r/68475/diff/2-3/


Testing
-------

ant clean test -Ds3.bucket.url=<bucket-url> -Ds3.generator.command=<credential-generator-command>


Thanks,

Boglarka Egyed


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message