sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bilung Lee" <bleeapa...@gmail.com>
Subject Review Request: SQOOP-314 Basic export hangs when target database does not support INSERT syntax with multiple rows of values
Date Thu, 18 Aug 2011 17:33:01 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1585/
-----------------------------------------------------------

Review request for Sqoop, Arvind Prabhakar and jmhsieh.


Summary
-------

Basic export job will hang when the target database does not support insert syntax with multiple
rows of values, such as INSERT INTO tbl (col1, col2) VALUES(11, 12),(21, 22),(23, 24)

This is because, in close(), AsyncSqlRecordWriter will still wait for AsyncSqlExecThread to
finish even when an SQLException is thrown underneath.

The fix is to introduce a new "--batch" option to execute underlying statements in batch mode
instead of a single multirow insert statement.


This addresses bugs Basic, INSERT, SQOOP-314, database, does, export, hangs, multiple, not,
of, rows, support, syntax, target, values, when, and with.
    https://issues.apache.org/jira/browse/Basic
    https://issues.apache.org/jira/browse/INSERT
    https://issues.apache.org/jira/browse/SQOOP-314
    https://issues.apache.org/jira/browse/database
    https://issues.apache.org/jira/browse/does
    https://issues.apache.org/jira/browse/export
    https://issues.apache.org/jira/browse/hangs
    https://issues.apache.org/jira/browse/multiple
    https://issues.apache.org/jira/browse/not
    https://issues.apache.org/jira/browse/of
    https://issues.apache.org/jira/browse/rows
    https://issues.apache.org/jira/browse/support
    https://issues.apache.org/jira/browse/syntax
    https://issues.apache.org/jira/browse/target
    https://issues.apache.org/jira/browse/values
    https://issues.apache.org/jira/browse/when
    https://issues.apache.org/jira/browse/with


Diffs
-----

  src/docs/man/sqoop-export.txt 6090ea1 
  src/docs/user/export.txt 4f87886 
  src/java/com/cloudera/sqoop/SqoopOptions.java d760d39 
  src/java/com/cloudera/sqoop/mapreduce/AsyncSqlRecordWriter.java 193cf41 
  src/java/com/cloudera/sqoop/mapreduce/ExportBatchOutputFormat.java PRE-CREATION 
  src/java/com/cloudera/sqoop/mapreduce/ExportJobBase.java 9799e37 
  src/java/com/cloudera/sqoop/mapreduce/ExportOutputFormat.java d2a6cf6 
  src/java/com/cloudera/sqoop/tool/BaseSqoopTool.java 8f629f1 
  src/java/com/cloudera/sqoop/tool/ExportTool.java b4b1091 
  src/test/com/cloudera/sqoop/manager/JdbcMySQLExportTest.java 8687b0c 

Diff: https://reviews.apache.org/r/1585/diff


Testing
-------


Thanks,

Bilung


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message