sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bilung Lee" <bleeapa...@gmail.com>
Subject Re: Review Request: SQOOP-314 Basic export hangs when target database does not support INSERT syntax with multiple rows of values
Date Fri, 19 Aug 2011 00:49:12 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated 2011-08-19 00:49:12.862727)

Review request for Sqoop, Arvind Prabhakar and jmhsieh.


New patch that incorporates comments.


Basic export job will hang when the target database does not support insert syntax with multiple
rows of values, such as INSERT INTO tbl (col1, col2) VALUES(11, 12),(21, 22),(23, 24)

This is because, in close(), AsyncSqlRecordWriter will still wait for AsyncSqlExecThread to
finish even when an SQLException is thrown underneath.

The fix is to introduce a new "--batch" option to execute underlying statements in batch mode
instead of a single multirow insert statement.

This addresses bug SQOOP-314.

Diffs (updated)

  src/java/com/cloudera/sqoop/mapreduce/ExportOutputFormat.java d2a6cf6 
  src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 8f629f1 
  src/java/com/cloudera/sqoop/tool/ExportTool.java b4b1091 
  src/test/com/cloudera/sqoop/manager/JdbcMySQLExportTest.java 8687b0c 
  src/docs/man/sqoop-export.txt 6090ea1 
  src/docs/user/export.txt 4f87886 
  src/java/com/cloudera/sqoop/SqoopOptions.java d760d39 
  src/java/com/cloudera/sqoop/manager/OracleManager.java 6a55312 
  src/java/com/cloudera/sqoop/manager/SQLServerManager.java e1ce2af 
  src/java/com/cloudera/sqoop/mapreduce/AsyncSqlRecordWriter.java 193cf41 
  src/java/com/cloudera/sqoop/mapreduce/ExportBatchOutputFormat.java PRE-CREATION 
  src/java/com/cloudera/sqoop/mapreduce/ExportJobBase.java 9799e37 

Diff: https://reviews.apache.org/r/1585/diff




  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message