sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abraham Elmahrek" <...@cloudera.com>
Subject Re: Review Request 33104: SQOOP-Hive import with Parquet should append automatically
Date Thu, 07 May 2015 01:46:37 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/33104/#review82786
-----------------------------------------------------------



src/test/com/cloudera/sqoop/hive/TestHiveImport.java
<https://reviews.apache.org/r/33104/#comment133601>

    Why not just create a Set object and compare sets? It should be much less code.


- Abraham Elmahrek


On May 6, 2015, 5:49 a.m., Qian Xu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/33104/
> -----------------------------------------------------------
> 
> (Updated May 6, 2015, 5:49 a.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Bugs: SQOOP-2295
>     https://issues.apache.org/jira/browse/SQOOP-2295
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> Currently, an existing dataset will throw an exception. This differs from `--as-textfile`.
I've checked the user manual, the handling of HDFS and Hive are indeed different. For HDFS,
unless `--append` is specified, the job will fail when destination exists already. For Hive,
unless `--create-hive-table` is specified, the job will become append mode. The patch has
made the handling of `--as-textfile` and `--as-parquetfile` consistent.
> 
> 
> Diffs
> -----
> 
>   src/docs/man/hive-args.txt 7d9e427 
>   src/docs/man/sqoop-create-hive-table.txt 7aebcc1 
>   src/docs/user/create-hive-table.txt 3aa34fd 
>   src/docs/user/hive-args.txt 53de92d 
>   src/java/org/apache/sqoop/mapreduce/DataDrivenImportJob.java d5bfae2 
>   src/java/org/apache/sqoop/mapreduce/ParquetJob.java df55dbc 
>   src/test/com/cloudera/sqoop/TestParquetImport.java 07e140a 
>   src/test/com/cloudera/sqoop/hive/TestHiveImport.java fa717cb 
>   src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java 7934791 
>   src/test/com/cloudera/sqoop/testutil/ImportJobTestCase.java 293bf10 
>   testdata/hive/scripts/normalImportAsParquet.q e434e9b 
> 
> Diff: https://reviews.apache.org/r/33104/diff/
> 
> 
> Testing
> -------
> 
> Manually tested append, new create and overwrite cases.
> 
> 
> Thanks,
> 
> Qian Xu
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message