sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick White" <nwh...@palantir.com>
Subject Re: Review Request: SQOOP-749: Exports Using Stored Procedures (Functions)
Date Thu, 17 Jan 2013 14:25:39 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8806/
-----------------------------------------------------------

(Updated Jan. 17, 2013, 2:25 p.m.)


Review request for Sqoop and Jarek Cecho.


Changes
-------

Sorry about the import stuff - I haven't figured out how to make eclipse remove unused imports
without sorting the remaining ones alphabetically (or how to be as productive in a different
editor :). The attached patch fixes all the issues you've raised. Thanks!


Description
-------

It'd be useful if you could use stored procedures (or functions) to insert data - currently
you can only use insert or update statements (or upsert / merges, depending on the SqlManager
you're using). This would help sqoop adoption / migration into environments which have existing,
SQL-based data import workflows. 

The attached patch adds a --call argument to the export tool that can be used in place of
--table.

Note that this patch depends on HSQLDB 2.x, whereas trunk currently depends on 1.8, so it
can't be committed until that situation's resolved (see SQOOP-519 and linked tickets). 


This addresses bug SQOOP-749.
    https://issues.apache.org/jira/browse/SQOOP-749


Diffs (updated)
-----

  ivy.xml 1ee60df 
  ivy/libraries.properties 4c9e37d 
  src/docs/user/export-purpose.txt c26eaa7 
  src/docs/user/export.txt 9f600fe 
  src/java/org/apache/sqoop/SqoopOptions.java b0fdfa0 
  src/java/org/apache/sqoop/manager/ConnManager.java 115186f 
  src/java/org/apache/sqoop/manager/SqlManager.java 3a52c6d 
  src/java/org/apache/sqoop/mapreduce/ExportCallOutputFormat.java PRE-CREATION 
  src/java/org/apache/sqoop/mapreduce/JdbcCallExportJob.java PRE-CREATION 
  src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java 00bd910 
  src/java/org/apache/sqoop/orm/ClassWriter.java 47e1221 
  src/java/org/apache/sqoop/tool/BaseSqoopTool.java b4b2213 
  src/java/org/apache/sqoop/tool/ExportTool.java acd296d 
  src/test/com/cloudera/sqoop/SmokeTests.java 76df6cf 
  src/test/com/cloudera/sqoop/TestConnFactory.java 893b388 
  src/test/com/cloudera/sqoop/TestExport.java eba10aa 
  src/test/com/cloudera/sqoop/manager/PostgresqlExportTest.java be449e4 
  src/test/com/cloudera/sqoop/testutil/ExportJobTestCase.java 4f6fd37 
  src/test/org/apache/sqoop/TestExportUsingProcedure.java PRE-CREATION 

Diff: https://reviews.apache.org/r/8806/diff/


Testing
-------

I've added a subclass of TestExport to make sure all the usual export functionality works.
I've also used it on live data exporting to postgres.


Thanks,

Nick White


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message