sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Masatake Iwasaki" <iwasak...@oss.nttdata.co.jp>
Subject Review Request 27187: SQOOP-1622: Copying from staging table should be in single transaction for pg_bulkload connector
Date Sat, 25 Oct 2014 02:09:49 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27187/
-----------------------------------------------------------

Review request for Sqoop.


Bugs: SQOOP-1622
    https://issues.apache.org/jira/browse/SQOOP-1622


Repository: sqoop-trunk


Description
-------

Each reduce tasks should do copying from staging table in single transaction.


Diffs
-----

  src/java/org/apache/sqoop/mapreduce/postgresql/PGBulkloadExportJob.java 32fe077 
  src/java/org/apache/sqoop/mapreduce/postgresql/PGBulkloadExportMapper.java 16af84c 
  src/test/com/cloudera/sqoop/manager/PGBulkloadManagerManualTest.java fc5fd6d 

Diff: https://reviews.apache.org/r/27187/diff/


Testing
-------

PGBulkloadExportReducer#reduce may be called per map task because the map output key is unique
ID of map task. This patch fixed the map output key to make reduce tasks do work in single
transaction.


Thanks,

Masatake Iwasaki


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message