sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venkatesh Seetharam (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SQOOP-638) Add an optional, simple and extensible validation framework for sqoop
Date Sat, 01 Dec 2012 20:43:58 GMT

     [ https://issues.apache.org/jira/browse/SQOOP-638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Venkatesh Seetharam updated SQOOP-638:

    Attachment: SQOOP-638-r6.patch
> Add an optional, simple and extensible validation framework for sqoop
> ---------------------------------------------------------------------
>                 Key: SQOOP-638
>                 URL: https://issues.apache.org/jira/browse/SQOOP-638
>             Project: Sqoop
>          Issue Type: Improvement
>    Affects Versions: 1.4.2
>            Reporter: Venkatesh Seetharam
>            Assignee: Venkatesh Seetharam
>            Priority: Minor
>             Fix For: 1.4.2
>         Attachments: SQOOP-638.patch, SQOOP-638-r6.patch
> Attempt to add an extensible validation framework to Sqoop. Adds an optional CLI option:
> There are 3 basic interfaces:
> ValidationThreshold - Determines if the error margin between the source and target are
acceptable: Absolute, Percentage Tolerant, etc.
> Default implementation is AbsoluteValidationThreshold which ensures the row counts from
source and targets are the same.
> ValidationFailureHandler - Responsible for handling failures: log an error/warning, abort,
etc. Default implementation logs a warning message to the configured logger.
> Validator - Drives the validation logic by delegating the decision to ValidationThreshold
and delegating failure handling to ValidationFailureHandler. The default implementation comes
with a RowCountValidator which validates the row counts from source and the target.
> You could extend these interfaces for more specific implementations and override 'em
in sqoop configuration that is picked up.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message