sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bilung Lee" <bleeapa...@gmail.com>
Subject Re: Review Request: SQOOP-314 Basic export hangs when target database does not support INSERT syntax with multiple rows of values
Date Thu, 18 Aug 2011 17:46:40 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated 2011-08-18 17:46:40.735309)

Review request for Sqoop, Arvind Prabhakar and jmhsieh.


Basic export job will hang when the target database does not support insert syntax with multiple
rows of values, such as INSERT INTO tbl (col1, col2) VALUES(11, 12),(21, 22),(23, 24)

This is because, in close(), AsyncSqlRecordWriter will still wait for AsyncSqlExecThread to
finish even when an SQLException is thrown underneath.

The fix is to introduce a new "--batch" option to execute underlying statements in batch mode
instead of a single multirow insert statement.

This addresses bug SQOOP-314.


  src/docs/man/sqoop-export.txt 6090ea1 
  src/docs/user/export.txt 4f87886 
  src/java/com/cloudera/sqoop/SqoopOptions.java d760d39 
  src/java/com/cloudera/sqoop/mapreduce/AsyncSqlRecordWriter.java 193cf41 
  src/java/com/cloudera/sqoop/mapreduce/ExportBatchOutputFormat.java PRE-CREATION 
  src/java/com/cloudera/sqoop/mapreduce/ExportJobBase.java 9799e37 
  src/java/com/cloudera/sqoop/mapreduce/ExportOutputFormat.java d2a6cf6 
  src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 8f629f1 
  src/java/com/cloudera/sqoop/tool/ExportTool.java b4b1091 
  src/test/com/cloudera/sqoop/manager/JdbcMySQLExportTest.java 8687b0c 

Diff: https://reviews.apache.org/r/1585/diff




  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message