sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Mazak" <pmazak2...@yahoo.com>
Subject Re: Review Request 34104: Fix for SQOOP-1369 Avro export ignores --columns option
Date Mon, 03 Aug 2015 18:41:09 GMT


> On July 7, 2015, 12:21 a.m., Abraham Elmahrek wrote:
> > src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java, line 104
> > <https://reviews.apache.org/r/34104/diff/2/?file=966944#file966944line104>
> >
> >     Why not set this to "null" in some cases instead? Then the below condition could
be:
> >     if (specifiedColumns == null || specifiedColumns.contains())...

Fixed. Also made SqlManager.filterSpecifiedColumnNames not use .isEmpty() for consistency.
Specified --columns option will have stuff in it or be null. Args parser does not allow empty.
Also, in order to test faster, added JdbcExportJobTest.


- Paul


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34104/#review90611
-----------------------------------------------------------


On July 7, 2015, 8:03 p.m., Paul Mazak wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34104/
> -----------------------------------------------------------
> 
> (Updated July 7, 2015, 8:03 p.m.)
> 
> 
> Review request for Sqoop and Yibing Shi.
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> Supplied a patch that worked for us. 
> There are 2 global places to consider - parent class SqlManager and child class CatalogQueryManager
- when returning getColumnNames to actually rely on those column names you specified with
--columns option. 
> There was 1 place to consider for Avro/Parquet in JdbcExportJob.configureGenericRecordExportInputFormat()
to only add the columnType if it's one that was specified with --columns option.
> 
> 
> Diffs
> -----
> 
>   src/java/org/apache/sqoop/manager/CatalogQueryManager.java 4e063ed 
>   src/java/org/apache/sqoop/manager/OracleManager.java 69b613f 
>   src/java/org/apache/sqoop/manager/SqlManager.java ead581d 
>   src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java 93d438a 
>   src/test/org/apache/sqoop/manager/SqlManagerTest.java PRE-CREATION 
>   src/test/org/apache/sqoop/mapreduce/JdbcExportJobTest.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/34104/diff/
> 
> 
> Testing
> -------
> 
> Integration testing passed exporting AVRO to MS SQL Server
> 
> 
> Thanks,
> 
> Paul Mazak
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message