sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick White" <nwh...@palantir.com>
Subject Re: Review Request: SQOOP-749: Exports Using Stored Procedures (Functions)
Date Thu, 17 Jan 2013 14:25:39 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated Jan. 17, 2013, 2:25 p.m.)

Review request for Sqoop and Jarek Cecho.


Sorry about the import stuff - I haven't figured out how to make eclipse remove unused imports
without sorting the remaining ones alphabetically (or how to be as productive in a different
editor :). The attached patch fixes all the issues you've raised. Thanks!


It'd be useful if you could use stored procedures (or functions) to insert data - currently
you can only use insert or update statements (or upsert / merges, depending on the SqlManager
you're using). This would help sqoop adoption / migration into environments which have existing,
SQL-based data import workflows. 

The attached patch adds a --call argument to the export tool that can be used in place of

Note that this patch depends on HSQLDB 2.x, whereas trunk currently depends on 1.8, so it
can't be committed until that situation's resolved (see SQOOP-519 and linked tickets). 

This addresses bug SQOOP-749.

Diffs (updated)

  ivy.xml 1ee60df 
  ivy/libraries.properties 4c9e37d 
  src/docs/user/export-purpose.txt c26eaa7 
  src/docs/user/export.txt 9f600fe 
  src/java/org/apache/sqoop/SqoopOptions.java b0fdfa0 
  src/java/org/apache/sqoop/manager/ConnManager.java 115186f 
  src/java/org/apache/sqoop/manager/SqlManager.java 3a52c6d 
  src/java/org/apache/sqoop/mapreduce/ExportCallOutputFormat.java PRE-CREATION 
  src/java/org/apache/sqoop/mapreduce/JdbcCallExportJob.java PRE-CREATION 
  src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java 00bd910 
  src/java/org/apache/sqoop/orm/ClassWriter.java 47e1221 
  src/java/org/apache/sqoop/tool/BaseSqoopTool.java b4b2213 
  src/java/org/apache/sqoop/tool/ExportTool.java acd296d 
  src/test/com/cloudera/sqoop/SmokeTests.java 76df6cf 
  src/test/com/cloudera/sqoop/TestConnFactory.java 893b388 
  src/test/com/cloudera/sqoop/TestExport.java eba10aa 
  src/test/com/cloudera/sqoop/manager/PostgresqlExportTest.java be449e4 
  src/test/com/cloudera/sqoop/testutil/ExportJobTestCase.java 4f6fd37 
  src/test/org/apache/sqoop/TestExportUsingProcedure.java PRE-CREATION 

Diff: https://reviews.apache.org/r/8806/diff/


I've added a subclass of TestExport to make sure all the usual export functionality works.
I've also used it on live data exporting to postgres.


Nick White

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message