sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Veena Basavaraj (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-1666) Sqoop2: JobManager code on failure should call SqoopDestroyerExecutor
Date Tue, 04 Nov 2014 19:30:36 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196656#comment-14196656
] 

Veena Basavaraj commented on SQOOP-1666:
----------------------------------------

did not know how critical matching is, but going forward we would consolidate this code in
one place,

marked it for 1.99.5

> Sqoop2: JobManager code  on failure should call SqoopDestroyerExecutor
> ----------------------------------------------------------------------
>
>                 Key: SQOOP-1666
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1666
>             Project: Sqoop
>          Issue Type: Bug
>            Reporter: Veena Basavaraj
>            Assignee: Veena Basavaraj
>             Fix For: 1.99.5
>
>
> There is more to just calling the destroyer.destroy method in the SqoopDestroyerExecutor
> Should not the code in JobManger call this utility class?
> The matching code is missing in the JobManager
> {code}
>  public static void executeDestroyer(boolean success, Configuration configuration, Direction
direction) {
>     String destroyerPropertyName, prefixPropertyName;
>     switch (direction) {
>       default:
>       case FROM:
>         destroyerPropertyName = MRJobConstants.JOB_ETL_FROM_DESTROYER;
>         prefixPropertyName = MRJobConstants.PREFIX_CONNECTOR_FROM_CONTEXT;
>         break;
>       case TO:
>         destroyerPropertyName = MRJobConstants.JOB_ETL_TO_DESTROYER;
>         prefixPropertyName = MRJobConstants.PREFIX_CONNECTOR_TO_CONTEXT;
>         break;
>     }
>     Destroyer destroyer = (Destroyer) ClassUtils.instantiate(configuration.get(destroyerPropertyName));
>     if(destroyer == null) {
>       LOG.info("Skipping running destroyer as non was defined.");
>       return;
>     }
>     // Objects that should be pass to the Destroyer execution
>     PrefixContext subContext = new PrefixContext(configuration, prefixPropertyName);
>     Object configConnection = MRConfigurationUtils.getConnectorConnectionConfig(direction,
configuration);
>     Object configJob = MRConfigurationUtils.getConnectorJobConfig(direction, configuration);
>     // Propagate connector schema in every case for now
>     Matcher matcher = MatcherFactory.getMatcher(
>         MRConfigurationUtils.getConnectorSchema(Direction.FROM, configuration),
>         MRConfigurationUtils.getConnectorSchema(Direction.TO, configuration));
>     Schema schema = direction == Direction.FROM ?
>         matcher.getFromSchema() : matcher.getToSchema();
>     DestroyerContext destroyerContext = new DestroyerContext(subContext, success, schema);
>     LOG.info("Executing destroyer class " + destroyer.getClass());
>     destroyer.destroy(destroyerContext, configConnection, configJob);
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message