sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick White" <nwh...@palantir.com>
Subject Re: Review Request: SQOOP-749: Exports Using Stored Procedures (Functions)
Date Thu, 10 Jan 2013 18:25:34 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated Jan. 10, 2013, 6:25 p.m.)

Review request for Sqoop and Jarek Cecho.


I've added third-party postgres tests to this latest revision. I don't have a MSSQL installation
around to test on, but if I add a --call test case to TestExport it'll get picked up by the
Oracle, JdbcMySQL & DirectMySQL tests. Would this be a good next step?


It'd be useful if you could use stored procedures (or functions) to insert data - currently
you can only use insert or update statements (or upsert / merges, depending on the SqlManager
you're using). This would help sqoop adoption / migration into environments which have existing,
SQL-based data import workflows. 

The attached patch adds a --call argument to the export tool that can be used in place of

Note that this patch depends on HSQLDB 2.x, whereas trunk currently depends on 1.8, so it
can't be committed until that situation's resolved (see SQOOP-519 and linked tickets). 

This addresses bug SQOOP-749.

Diffs (updated)

  ivy.xml 1ee60df 
  ivy/libraries.properties 4c9e37d 
  src/docs/user/export-purpose.txt c26eaa7 
  src/docs/user/export.txt 9f600fe 
  src/java/org/apache/sqoop/SqoopOptions.java 3e0ec3e 
  src/java/org/apache/sqoop/manager/ConnManager.java 21eea93 
  src/java/org/apache/sqoop/manager/SqlManager.java 3a52c6d 
  src/java/org/apache/sqoop/mapreduce/ExportCallOutputFormat.java PRE-CREATION 
  src/java/org/apache/sqoop/mapreduce/JdbcCallExportJob.java PRE-CREATION 
  src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java 7c52110 
  src/java/org/apache/sqoop/orm/ClassWriter.java b73711e 
  src/java/org/apache/sqoop/tool/BaseSqoopTool.java c0221c9 
  src/java/org/apache/sqoop/tool/ExportTool.java acd296d 
  src/test/com/cloudera/sqoop/SmokeTests.java 76df6cf 
  src/test/com/cloudera/sqoop/TestConnFactory.java 893b388 
  src/test/com/cloudera/sqoop/TestExport.java eba10aa 
  src/test/com/cloudera/sqoop/manager/PostgresqlExportTest.java be449e4 
  src/test/com/cloudera/sqoop/testutil/ExportJobTestCase.java 4f6fd37 
  src/test/org/apache/sqoop/TestExportUsingProcedure.java PRE-CREATION 

Diff: https://reviews.apache.org/r/8806/diff/


I've added a subclass of TestExport to make sure all the usual export functionality works.
I've also used it on live data exporting to postgres.


Nick White

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message