sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Mazak" <pmazak2...@yahoo.com>
Subject Re: Review Request 34104: Fix for SQOOP-1369 Avro export ignores --columns option
Date Mon, 03 Aug 2015 18:40:38 GMT


> On July 22, 2015, 9:57 p.m., Jarek Cecho wrote:
> > I have few more comments:
> > 
> > 1) The JIRA specifies that the user visible problem is that export from Avro won't
work correctly. Can we add test for that, so that we know that the problem that we're trying
to address is actually addressed? :)

Done. Before, I was testing the method Avro relies on - JdbcExportJob.configureGenericRecordExportInputFormat
- implicitly. Now in the test, I set inputFileType to Avro and go through JdbcExportJob.configureInputFormat
explicitly.


- Paul


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34104/#review92659
-----------------------------------------------------------


On July 7, 2015, 8:03 p.m., Paul Mazak wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/34104/
> -----------------------------------------------------------
> 
> (Updated July 7, 2015, 8:03 p.m.)
> 
> 
> Review request for Sqoop and Yibing Shi.
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> Supplied a patch that worked for us. 
> There are 2 global places to consider - parent class SqlManager and child class CatalogQueryManager
- when returning getColumnNames to actually rely on those column names you specified with
--columns option. 
> There was 1 place to consider for Avro/Parquet in JdbcExportJob.configureGenericRecordExportInputFormat()
to only add the columnType if it's one that was specified with --columns option.
> 
> 
> Diffs
> -----
> 
>   src/java/org/apache/sqoop/manager/CatalogQueryManager.java 4e063ed 
>   src/java/org/apache/sqoop/manager/OracleManager.java 69b613f 
>   src/java/org/apache/sqoop/manager/SqlManager.java ead581d 
>   src/java/org/apache/sqoop/mapreduce/JdbcExportJob.java 93d438a 
>   src/test/org/apache/sqoop/manager/SqlManagerTest.java PRE-CREATION 
>   src/test/org/apache/sqoop/mapreduce/JdbcExportJobTest.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/34104/diff/
> 
> 
> Testing
> -------
> 
> Integration testing passed exporting AVRO to MS SQL Server
> 
> 
> Thanks,
> 
> Paul Mazak
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message