sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Attila Szabo <asz...@cloudera.com>
Subject Re: Review Request 50155: SQOOP-2983: OraOop export has degraded performance with wide tables
Date Tue, 09 Aug 2016 22:50:27 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50155/
-----------------------------------------------------------

(Updated Aug. 9, 2016, 10:50 p.m.)


Review request for Sqoop, David Robson, Jarek Cecho, and Kathleen Ting.


Changes
-------

With the help of David Robson I was able to identify one issue around "update-key" option,
and also was able to spot another issue (left behind after the changes around Oracle escaped
column name support). Both of them are fixed. New test case attached as well. New diff reflects
all of the changes. Please do another round of review!


Repository: sqoop-trunk


Description
-------

Proposed changes for SQOOP-2983


Diffs (updated)
-----

  src/java/org/apache/sqoop/manager/oracle/OraOopConnManager.java 216c771 
  src/java/org/apache/sqoop/manager/oracle/OraOopOracleQueries.java 6b27bd8 
  src/java/org/apache/sqoop/manager/oracle/OraOopOutputFormatBase.java 8f94cf8 
  src/java/org/apache/sqoop/manager/oracle/OraOopOutputFormatInsert.java d5eebf4 
  src/java/org/apache/sqoop/manager/oracle/OraOopOutputFormatUpdate.java a33768f 
  src/java/org/apache/sqoop/manager/oracle/OraOopUtilities.java e81588c 
  src/java/org/apache/sqoop/orm/ClassWriter.java 9d91887 
  src/test/org/apache/sqoop/manager/oracle/ExportTest.java 991b221 
  src/test/org/apache/sqoop/manager/oracle/OraOopTestCase.java 9fe4821 
  src/test/org/apache/sqoop/manager/oracle/OraOopTypesTest.java PRE-CREATION 
  src/test/org/apache/sqoop/manager/oracle/OraOopUpdateKeyTest.java PRE-CREATION 

Diff: https://reviews.apache.org/r/50155/diff/


Testing
-------

800 columns with table
100.000 lines (156mb data)
1.000.000 lines (1.56 GB data)
3.000.000 lines (4.5 GB data)


Thanks,

Attila Szabo


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message