sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Veena Basavaraj (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-1652) Submit/Start terminology and SubmissionStatus enums in client and server clarifications
Date Thu, 06 Nov 2014 22:52:34 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201117#comment-14201117

Veena Basavaraj commented on SQOOP-1652:

[~jarcec] we need to address the inconsistencies in the code.

I am not very thrilled about start, since "start" usually does not start the job, But if we
are going to keep this the Driver Error and status STATES should be revisited once to reflect
the consistent usage

> Submit/Start terminology and SubmissionStatus enums in client and server clarifications
> ---------------------------------------------------------------------------------------
>                 Key: SQOOP-1652
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1652
>             Project: Sqoop
>          Issue Type: Bug
>            Reporter: Veena Basavaraj
>            Assignee: Veena Basavaraj
>             Fix For: 2.0.0
> This ticket would like to address 2 broader questions.
> What is the terminology we would want to use for job execution process. 
> There are 2 words very inconsistently used in the code base. Submit/ Start in fact mean
2 different things, there are client/ server apis that use START, but there are error messages
and enums communicating status as SUBMITTED in the client and STARTED in other places. 
> Second, Some discussions have alluded that client and the rest API are different? I want
to challenge this and ask why ? What is the reasoning for it to be different. It just adds
a lot of overhead to think about 2 of these growing apart differently, as a long term contributor
to this project, I'd like to minimize this deviation if there is no solid reason to have it
that way. To some folks this might look a bit trivial but it will help us not having to debate
about things every time we are in this area adding new features. 
> the following using the SERVER status enum
> {code}
> @SuppressWarnings("static-access")
>   public ShowJobStatusFunction() {
>     this.addOption(OptionBuilder.hasArg().withArgName(Constants.OPT_JID)
>        .withDescription(resourceString(Constants.RES_PROMPT_JOB_ID))
>        .withLongOpt(Constants.OPT_JID)
>        .create(Constants.OPT_JID_CHAR));
>   }
>   @Override
>   public Object executeFunction(CommandLine line, boolean isInteractive) {
>     if (line.hasOption(Constants.OPT_JID)) {
>       MSubmission submission = client.getJobStatus(getLong(line, Constants.OPT_JID));
>       if(submission.getStatus().isFailure() || submission.getStatus().equals(SubmissionStatus.SUCCEEDED))
>         SubmissionDisplayer.displayHeader(submission);
>         SubmissionDisplayer.displayFooter(submission);
>       } else {
>         SubmissionDisplayer.displayHeader(submission);
>         SubmissionDisplayer.displayProgress(submission);
>       }
>     } else {
>       return null;
>     }
> {code}
> the following startJob in cleint code calls the server status and then uses its OWN enum
to decide what to show.
> {code}
>  MSubmission submission = resourceRequests.startJob(jobId).getSubmissions().get(0);
>     while(submission.getStatus().isRunning()) {
>       if(submitted) {
>         invokeSubmissionCallback(callback, submission, SubmissionStatus.SUBMITTED);
>         submitted = false;
>       } else {
>         invokeSubmissionCallback(callback, submission, SubmissionStatus.UPDATED);
>       }
>       Thread.sleep(pollTime);
>       submission = getJobStatus(jobId);
>     }
>     invokeSubmissionCallback(callback, submission, SubmissionStatus.FINISHED);
>     return submission;
>   }
> {code}
> But not having a consistent view of things will lead to confusion down the line when
new features are added.
> Similarly we need have our design philosophy around concrete on other things in the code
> 1. How we distinguish between the connector and execution engine responsibilities 
> 2. How we think of the sqoop internal/core behavior vs the API behavior. 
> 3. Repository API: How should it evolve? Should I expose an API to allow nulls in the
SQI_INPUT names or should I enforce this in the SQOOP code and not leave it to the decision
of the repository implementation
> Today we have unique name constraints in the Derby. How can we enforce such constraints
are included in every new implementation we add ? 

This message was sent by Atlassian JIRA

View raw message