sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Masatake Iwasaki" <iwasak...@oss.nttdata.co.jp>
Subject Re: Review Request 27187: SQOOP-1622: Copying from staging table should be in single transaction for pg_bulkload connector
Date Mon, 03 Nov 2014 18:08:34 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27187/
-----------------------------------------------------------

(Updated Nov. 3, 2014, 6:08 p.m.)


Review request for Sqoop.


Bugs: SQOOP-1622
    https://issues.apache.org/jira/browse/SQOOP-1622


Repository: sqoop-trunk


Description (updated)
-------

PGBulkloadExportReducer#reduce may be called per map task because the map output key is unique
ID of map task. This patch fixed the map output key to make reduce tasks do work in single
transaction.
This patch include trivial fixes in test code.
Crutial part this patch is just moving the call conn.commit() in PGBulkloadExportReducer.



Diffs
-----

  src/java/org/apache/sqoop/mapreduce/postgresql/PGBulkloadExportJob.java 32fe077 
  src/java/org/apache/sqoop/mapreduce/postgresql/PGBulkloadExportMapper.java 16af84c 
  src/java/org/apache/sqoop/mapreduce/postgresql/PGBulkloadExportReducer.java 3dc05a7 
  src/test/com/cloudera/sqoop/manager/PGBulkloadManagerManualTest.java fc5fd6d 

Diff: https://reviews.apache.org/r/27187/diff/


Testing (updated)
-------

PGBulkloadManagerManualTest passed.

I removed PGBulkloadManagerManualTest#multiFileTestWithGenericArgs because TestExport#multiFileTest
can handle generic args as far as args are given in the form as not "-Dvar=val" but "-D" and
"var=val".


Thanks,

Masatake Iwasaki


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message