sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick White" <nwh...@palantir.com>
Subject Review Request: SQOOP-749: Exports Using Stored Procedures (Functions)
Date Thu, 03 Jan 2013 00:36:09 GMT

This is an automatically generated e-mail. To reply, visit:

Review request for Sqoop and Jarek Cecho.


It'd be useful if you could use stored procedures (or functions) to insert data - currently
you can only use insert or update statements (or upsert / merges, depending on the SqlManager
you're using). This would help sqoop adoption / migration into environments which have existing,
SQL-based data import workflows. 

The attached patch adds a --call argument to the export tool that can be used in place of

Note that this patch depends on HSQLDB 2.x, whereas trunk currently depends on 1.8, so it
can't be committed until that situation's resolved (see SQOOP-519 and linked tickets). 

This addresses bug SQOOP-749.


  src/java/org/apache/sqoop/SqoopOptions.java 613f797 
  src/java/org/apache/sqoop/manager/ConnManager.java 21eea93 
  src/java/org/apache/sqoop/manager/SqlManager.java 3a52c6d 
  src/java/org/apache/sqoop/mapreduce/ExportCallOutputFormat.java PRE-CREATION 
  src/java/org/apache/sqoop/mapreduce/JdbcCallExportJob.java PRE-CREATION 
  src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java 7c52110 
  src/java/org/apache/sqoop/orm/ClassWriter.java b73711e 
  src/java/org/apache/sqoop/tool/BaseSqoopTool.java d795646 
  src/java/org/apache/sqoop/tool/ExportTool.java acd296d 
  src/test/com/cloudera/sqoop/SmokeTests.java 76df6cf 
  src/test/com/cloudera/sqoop/TestConnFactory.java 893b388 
  src/test/com/cloudera/sqoop/TestExport.java eba10aa 
  src/test/com/cloudera/sqoop/testutil/ExportJobTestCase.java 4f6fd37 
  src/test/org/apache/sqoop/TestExportUsingProcedure.java PRE-CREATION 

Diff: https://reviews.apache.org/r/8806/diff/


I've added a subclass of TestExport to make sure all the usual export functionality works.
I've also used it on live data exporting to postgres.


Nick White

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message