sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Szabolcs Vasas <vasas.szabo...@gmail.com>
Subject Re: Review Request 37251: SQOOP-2457: Add option to automatically compute statistics after loading date into a hive table
Date Fri, 15 Jul 2016 08:07:26 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37251/#review142346
-----------------------------------------------------------


Ship it!




Ship It!

- Szabolcs Vasas


On Aug. 8, 2015, 4:09 a.m., Venkat Ranganathan wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37251/
> -----------------------------------------------------------
> 
> (Updated Aug. 8, 2015, 4:09 a.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Bugs: SQOOP-2457
>     https://issues.apache.org/jira/browse/SQOOP-2457
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> With CBO and different execution engines like Tez depedning on statistics like row count
heavily, it is important that we provide the option to update stats on data loaded into Hive
as part of the --hive-import option.  Ideally these should be Hive managed, but there are
use cases where this is not automatic and hence this option will help in those cases
> 
> Added a new option --hive-compute-stats which will add compute statistics statement for
the loaded table/partition as the case may be for --hive-imports
> 
> 
> Diffs
> -----
> 
>   src/java/org/apache/sqoop/SqoopOptions.java 9405605 
>   src/java/org/apache/sqoop/hive/HiveImport.java e03d33c 
>   src/java/org/apache/sqoop/hive/TableDefWriter.java c9962e9 
>   src/java/org/apache/sqoop/tool/BaseSqoopTool.java 4e2e66d 
>   src/test/com/cloudera/sqoop/hive/TestHiveImport.java b626964 
>   src/test/com/cloudera/sqoop/hive/TestTableDefWriter.java 55e572e 
>   testdata/hive/scripts/normalWithStatsImport.q PRE-CREATION 
>   testdata/hive/scripts/partitionWithStatsImport.q PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/37251/diff/
> 
> 
> Testing
> -------
> 
> Added new tests and all tests pass
> 
> 
> Thanks,
> 
> Venkat Ranganathan
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message