sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abraham Elmahrek" <...@cloudera.com>
Subject Re: Review Request 27187: SQOOP-1622: Copying from staging table should be in single transaction for pg_bulkload connector
Date Fri, 14 Nov 2014 23:54:40 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27187/#review61558
-----------------------------------------------------------

Ship it!


https://warosu.org/data/lit/img/0051/88/1406367367200.gif

- Abraham Elmahrek


On Nov. 3, 2014, 6:08 p.m., Masatake Iwasaki wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/27187/
> -----------------------------------------------------------
> 
> (Updated Nov. 3, 2014, 6:08 p.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Bugs: SQOOP-1622
>     https://issues.apache.org/jira/browse/SQOOP-1622
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> PGBulkloadExportReducer#reduce may be called per map task because the map output key
is unique ID of map task. This patch fixed the map output key to make reduce tasks do work
in single transaction.
> This patch include trivial fixes in test code.
> Crutial part this patch is just moving the call conn.commit() in PGBulkloadExportReducer.

> 
> 
> Diffs
> -----
> 
>   src/java/org/apache/sqoop/mapreduce/postgresql/PGBulkloadExportJob.java 32fe077 
>   src/java/org/apache/sqoop/mapreduce/postgresql/PGBulkloadExportMapper.java 16af84c

>   src/java/org/apache/sqoop/mapreduce/postgresql/PGBulkloadExportReducer.java 3dc05a7

>   src/test/com/cloudera/sqoop/manager/PGBulkloadManagerManualTest.java fc5fd6d 
> 
> Diff: https://reviews.apache.org/r/27187/diff/
> 
> 
> Testing
> -------
> 
> PGBulkloadManagerManualTest passed.
> 
> I removed PGBulkloadManagerManualTest#multiFileTestWithGenericArgs because TestExport#multiFileTest
can handle generic args as far as args are given in the form as not "-Dvar=val" but "-D" and
"var=val".
> 
> 
> Thanks,
> 
> Masatake Iwasaki
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message