sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Abraham Elmahrek" <...@cloudera.com>
Subject Re: Review Request 31749: SQOOP-2164: Enhance the Netezza Connector for Sqoop to support new options and copy logs to HDFS
Date Thu, 05 Mar 2015 22:01:33 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/31749/#review75371
-----------------------------------------------------------


Good stuff


src/java/org/apache/sqoop/manager/DirectNetezzaManager.java
<https://reviews.apache.org/r/31749/#comment122374>

    `conf.setBoolean(NETEZZA_CTRL_CHARS_OPT, cmdLine.hasOption(NETEZZA_CTRL_CHARS_LONG_ARG));`
    
    Same for next one as well.



src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableExportMapper.java
<https://reviews.apache.org/r/31749/#comment122426>

    Seems we do this twice. Possible to pull out into a utility method?



src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableImportMapper.java
<https://reviews.apache.org/r/31749/#comment122427>

    See comment about utility class above



src/java/org/apache/sqoop/util/FileUploader.java
<https://reviews.apache.org/r/31749/#comment122417>

    `if (!fs.exists(targetPath)) fs.mkdirs(targetPath)?`
    
    Or something similar to https://github.com/apache/hadoop/blob/branch-1.2/src/test/org/apache/hadoop/hdfs/TestDFSMkdirs.java#L51
or https://github.com/apache/hadoop/blob/branch-2.3/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSMkdirs.java#L62



src/java/org/apache/sqoop/util/FileUploader.java
<https://reviews.apache.org/r/31749/#comment122424>

    Err confusing... did find this: http://stackoverflow.com/questions/12783968/copying-directory-from-local-system-to-hdfs-java-code.
Maybe add a comment?


- Abraham Elmahrek


On March 5, 2015, 12:13 a.m., Venkat Ranganathan wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/31749/
> -----------------------------------------------------------
> 
> (Updated March 5, 2015, 12:13 a.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> The following options are enabled:  ctrlchars, truncstring
> 
> Also, now log-dir option is used to copy the generated logs from the Netezza external
table operation to the context filesystem so that they can be easily perused
> 
> Documentation has been updated
> 
> 
> Diffs
> -----
> 
>   src/docs/user/connectors.txt fee40d9 
>   src/java/org/apache/sqoop/manager/DirectNetezzaManager.java 9ca8f63 
>   src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableExportMapper.java
3613ff2 
>   src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableImportMapper.java
2f4c152 
>   src/java/org/apache/sqoop/util/FileUploader.java PRE-CREATION 
>   src/test/org/apache/sqoop/manager/netezza/DirectNetezzaExportManualTest.java f7b68ed

> 
> Diff: https://reviews.apache.org/r/31749/diff/
> 
> 
> Testing
> -------
> 
> Added tests to test for the new options.   Ran all the Netezza tests with this change
and all passed.   Ran manual tests on a cluster to make sure the new options work
> 
> 
> Thanks,
> 
> Venkat Ranganathan
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message