sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venkat Ranganathan" <n....@live.com>
Subject Re: Review Request: SQOOP-931 - Integration of Sqoop and HCatalog
Date Mon, 03 Jun 2013 04:16:49 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10688/
-----------------------------------------------------------

(Updated June 3, 2013, 4:16 a.m.)


Review request for Sqoop and Jarek Cecho.


Changes
-------

Same as previous one except fixed the trailing blanks in hcatalog.txt documentation.   No
real change to the rendered HTML.   Sorry for one more minor update


Description
-------

This patch implements the new feature of integrating HCatalog and Sqoop.   With this feature,
it is possible to import and export data between Sqoop and HCatalog tables.   The document
attached to SQOOP-931 JIRA issue discusses the high level appraches.  

With this integration, more fidelity can be brought to the process of moving data between
enterprise data stores and hadoop ecosystem.


Diffs (updated)
-----

  build.xml 636c103 
  ivy.xml 1fa4dd1 
  ivy/ivysettings.xml c4cc561 
  src/docs/user/SqoopUserGuide.txt 01ac1cf 
  src/docs/user/hcatalog.txt PRE-CREATION 
  src/java/org/apache/sqoop/SqoopOptions.java f18d43e 
  src/java/org/apache/sqoop/config/ConfigurationConstants.java 5354063 
  src/java/org/apache/sqoop/hive/HiveImport.java 838f083 
  src/java/org/apache/sqoop/manager/ConnManager.java a1ac38e 
  src/java/org/apache/sqoop/mapreduce/DataDrivenImportJob.java ef1d363 
  src/java/org/apache/sqoop/mapreduce/ExportJobBase.java 1065d0b 
  src/java/org/apache/sqoop/mapreduce/ImportJobBase.java 2465f3f 
  src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java 20636a0 
  src/java/org/apache/sqoop/mapreduce/JobBase.java 0df1156 
  src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportFormat.java PRE-CREATION 
  src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatExportMapper.java PRE-CREATION 
  src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatImportMapper.java PRE-CREATION 
  src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatInputSplit.java PRE-CREATION 
  src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatRecordReader.java PRE-CREATION 
  src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatUtilities.java PRE-CREATION 
  src/java/org/apache/sqoop/tool/BaseSqoopTool.java 42f521f 
  src/java/org/apache/sqoop/tool/CodeGenTool.java dd34a97 
  src/java/org/apache/sqoop/tool/ExportTool.java 215addd 
  src/java/org/apache/sqoop/tool/ImportTool.java 2627726 
  src/perftest/ExportStressTest.java 0a41408 
  src/test/com/cloudera/sqoop/hive/TestHiveImport.java 462ccf1 
  src/test/com/cloudera/sqoop/testutil/BaseSqoopTestCase.java cf41b96 
  src/test/com/cloudera/sqoop/testutil/ExportJobTestCase.java e13f3df 
  src/test/org/apache/sqoop/hcat/HCatalogExportTest.java PRE-CREATION 
  src/test/org/apache/sqoop/hcat/HCatalogImportTest.java PRE-CREATION 
  src/test/org/apache/sqoop/hcat/HCatalogTestUtils.java PRE-CREATION 
  src/test/org/apache/sqoop/hcat/TestHCatalogBasic.java PRE-CREATION 
  testdata/hcatalog/conf/hive-log4j.properties PRE-CREATION 
  testdata/hcatalog/conf/hive-site.xml PRE-CREATION 
  testdata/hcatalog/conf/log4j.properties PRE-CREATION 

Diff: https://reviews.apache.org/r/10688/diff/


Testing
-------

Two new integration test suites with more than 20 tests in total have been added to test various
aspects of the integration.  A unit test to test the option management is also added.   All
tests pass


Thanks,

Venkat Ranganathan


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message