sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Qian Xu" <qian.a...@intel.com>
Subject Re: Review Request 33104: SQOOP-Hive import with Parquet should append automatically
Date Sun, 03 May 2015 15:40:10 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated May 3, 2015, 11:40 p.m.)

Review request for Sqoop.


Removed `--create-hive-table` related code regarding Jarcec's comments.

Bugs: SQOOP-2295

Repository: sqoop-trunk


Currently, an existing dataset will throw an exception. This differs from `--as-textfile`.
I've checked the user manual, the handling of HDFS and Hive are indeed different. For HDFS,
unless `--append` is specified, the job will fail when destination exists already. For Hive,
unless `--create-hive-table` is specified, the job will become append mode. The patch has
made the handling of `--as-textfile` and `--as-parquetfile` consistent.

Diffs (updated)

  src/docs/man/hive-args.txt 7d9e427 
  src/docs/man/sqoop-create-hive-table.txt 7aebcc1 
  src/docs/user/create-hive-table.txt 3aa34fd 
  src/docs/user/hive-args.txt 53de92d 
  src/java/org/apache/sqoop/mapreduce/DataDrivenImportJob.java d5bfae2 
  src/java/org/apache/sqoop/mapreduce/ParquetJob.java df55dbc 
  src/test/com/cloudera/sqoop/hive/TestHiveImport.java fa717cb 
  src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java 7934791 
  testdata/hive/scripts/normalImportAsParquet.q e434e9b 

Diff: https://reviews.apache.org/r/33104/diff/


Manually tested append, new create and overwrite cases.


Qian Xu

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message