sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Boglarka Egyed <b...@apache.org>
Subject Re: Review Request 68536: SQOOP-3368: Add fail-fast scenarios to S3 incremental import use cases without --temporary-rootdir option
Date Wed, 29 Aug 2018 11:11:53 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated Aug. 29, 2018, 11:11 a.m.)

Review request for Sqoop, daniel voros, Fero Szabo, Nguyen Truong, and Szabolcs Vasas.

Bugs: SQOOP-3368

Repository: sqoop-trunk


The current implementation of Sqoop handles HDFS as a default filesystem, i.e. it creates
temporary directories on HDFS in case of incremental append or merge imports. To make these
incremental import use cases work with S3 the user needs to set the --temporary-rootdir to
an S3 location properly.


  src/java/org/apache/sqoop/tool/ImportTool.java 139733732d2a28d171568b9118c98a47a3d2fc50

  src/test/org/apache/sqoop/tool/TestS3IncrementalImportOptionValidations.java PRE-CREATION

Diff: https://reviews.apache.org/r/68536/diff/2/

Testing (updated)

ant clean test
./gradlew test -Ds3.bucket.url=<bucket-url> -Ds3.generator.command=<credential-generator-command>


Boglarka Egyed

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message