sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-638) Add an optional, simple and extensible validation framework for sqoop
Date Sat, 01 Dec 2012 21:31:58 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13508059#comment-13508059
] 

Hudson commented on SQOOP-638:
------------------------------

Integrated in Sqoop-ant-jdk-1.6-hadoop100 #347 (See [https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/347/])
    SQOOP-638: Add an optional, simple and extensible validation framework for sqoop (Revision
0b465594d24827c5a8d28e81ed3487e82937a72b)

     Result = SUCCESS
jarcec : https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=0b465594d24827c5a8d28e81ed3487e82937a72b
Files : 
* src/java/org/apache/sqoop/tool/BaseSqoopTool.java
* src/docs/user/import.txt
* src/java/org/apache/sqoop/tool/ImportTool.java
* src/java/org/apache/sqoop/validation/Validator.java
* src/docs/user/validation-args.txt
* src/java/com/cloudera/sqoop/mapreduce/JobBase.java
* src/java/org/apache/sqoop/validation/RowCountValidator.java
* src/docs/user/SqoopUserGuide.txt
* src/java/org/apache/sqoop/validation/AbsoluteValidationThreshold.java
* src/test/org/apache/sqoop/validation/RowCountValidatorImportTest.java
* src/java/org/apache/sqoop/validation/ValidationFailureHandler.java
* src/java/org/apache/sqoop/mapreduce/ImportJobBase.java
* src/java/org/apache/sqoop/mapreduce/ExportJobBase.java
* src/docs/user/common-args.txt
* src/java/org/apache/sqoop/validation/ValidationContext.java
* src/java/org/apache/sqoop/SqoopOptions.java
* src/java/org/apache/sqoop/validation/ValidationException.java
* src/java/org/apache/sqoop/validation/LogOnFailureHandler.java
* src/java/org/apache/sqoop/tool/ExportTool.java
* src/java/org/apache/sqoop/validation/ValidationThreshold.java
* src/docs/user/export.txt
* src/docs/user/validation.txt

                
> Add an optional, simple and extensible validation framework for sqoop
> ---------------------------------------------------------------------
>
>                 Key: SQOOP-638
>                 URL: https://issues.apache.org/jira/browse/SQOOP-638
>             Project: Sqoop
>          Issue Type: Improvement
>    Affects Versions: 1.4.2
>            Reporter: Venkatesh Seetharam
>            Assignee: Venkatesh Seetharam
>            Priority: Minor
>             Fix For: 1.4.3
>
>         Attachments: SQOOP-638.patch, SQOOP-638-r6.patch
>
>
> Attempt to add an extensible validation framework to Sqoop. Adds an optional CLI option:
--validate
> There are 3 basic interfaces:
> ValidationThreshold - Determines if the error margin between the source and target are
acceptable: Absolute, Percentage Tolerant, etc.
> Default implementation is AbsoluteValidationThreshold which ensures the row counts from
source and targets are the same.
> ValidationFailureHandler - Responsible for handling failures: log an error/warning, abort,
etc. Default implementation logs a warning message to the configured logger.
> Validator - Drives the validation logic by delegating the decision to ValidationThreshold
and delegating failure handling to ValidationFailureHandler. The default implementation comes
with a RowCountValidator which validates the row counts from source and the target.
> You could extend these interfaces for more specific implementations and override 'em
in sqoop configuration that is picked up.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message