sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Veena Basavaraj (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SQOOP-1879) Sqoop2: Submission Engine does not set all details on SubmissionRecord in Local mode
Date Tue, 16 Dec 2014 17:15:14 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-1879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14248424#comment-14248424
] 

Veena Basavaraj edited comment on SQOOP-1879 at 12/16/14 5:15 PM:
------------------------------------------------------------------

[~abec] the last patch has the lingering "updateSubmission" call, hence you saw those errors.

This one does have failures via local runner ( for FromRDBMSToHDFSTest) , but passes all the
cluster based integration tests

{code}
[INFO] Surefire report directory: /Users/vbasavaraj/Projects/sqoop2/test/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.sqoop.integration.connector.jdbc.generic.FromHDFSToRDBMSTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.808 sec
Running org.apache.sqoop.integration.connector.jdbc.generic.FromRDBMSToHDFSTest
Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 22.408 sec <<< FAILURE!
Running org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest
Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 196.276 sec
Running org.apache.sqoop.integration.connector.jdbc.generic.TableStagedRDBMSTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.217 sec
Running org.apache.sqoop.integration.connector.kafka.FromRDBMSToKafkaTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.683 sec
Running org.apache.sqoop.integration.repository.derby.upgrade.Derby1_99_4UpgradeTest
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 121.865 sec <<< FAILURE!
Running org.apache.sqoop.integration.server.SubmissionWithDisabledModelObjectsTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.176 sec
Running org.apache.sqoop.integration.server.VersionTest
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 8.595 sec <<< FAILURE!

Results :

Failed tests:   testColumns(org.apache.sqoop.integration.connector.jdbc.generic.FromRDBMSToHDFSTest)
  testVersion(org.apache.sqoop.integration.server.VersionTest): expected:<[Mon Dec 15 17:45:39]
PST 2014> but was:<[Sat Dec 13 18:20:35] PST 2014>

Tests in error: 
  testPostUpgrade(org.apache.sqoop.integration.repository.derby.upgrade.Derby1_99_4UpgradeTest):
Failed to start the Tomcat 6.x container. Check the [/Users/vbasavaraj/Projects/sqoop2/test/target/sqoop-cargo-tests/org.apache.sqoop.integration.repository.derby.upgrade.Derby1_99_4UpgradeTest/testPostUpgrade/sqoop-mini-cluster//log/tomcat.log]
file containing the container logs for more details.

Tests run: 30, Failures: 2, Errors: 1, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 07:16 min
[INFO] Finished at: 2014-12-16T08:22:55-08:00
[INFO] Final Memory: 70M/801M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.

{code}


mvn clean integration-test : Everything passes except Derby.

{code}
Running org.apache.sqoop.integration.connector.jdbc.generic.FromHDFSToRDBMSTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 59.747 sec
Running org.apache.sqoop.integration.connector.jdbc.generic.FromRDBMSToHDFSTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 86.5 sec
Running org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest
Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 692.193 sec
Running org.apache.sqoop.integration.connector.jdbc.generic.TableStagedRDBMSTest

Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 61.457 sec
Running org.apache.sqoop.integration.connector.kafka.FromRDBMSToKafkaTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 70.026 sec
Running org.apache.sqoop.integration.repository.derby.upgrade.Derby1_99_4UpgradeTest
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 121.817 sec <<< FAILURE!
Running org.apache.sqoop.integration.server.SubmissionWithDisabledModelObjectsTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 48.666 sec
Running org.apache.sqoop.integration.server.VersionTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.626 sec

Results :

Tests in error: 
  testPostUpgrade(org.apache.sqoop.integration.repository.derby.upgrade.Derby1_99_4UpgradeTest):
Failed to start the Tomcat 6.x container. Check the [/Users/vbasavaraj/Projects/sqoop2/test/target/sqoop-cargo-tests/org.apache.sqoop.integration.repository.derby.upgrade.Derby1_99_4UpgradeTest/testPostUpgrade/sqoop-mini-cluster//log/tomcat.log]
file containing the container logs for more details.

Tests run: 30, Failures: 0, Errors: 1, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] Sqoop .............................................. SUCCESS [  0.535 s]
[INFO] Sqoop Common ....................................... SUCCESS [ 21.087 s]
[INFO] Sqoop Common Test Libraries ........................ SUCCESS [  0.766 s]
[INFO] Sqoop Connectors ................................... SUCCESS [  0.012 s]
[INFO] Sqoop Connector SDK ................................ SUCCESS [  1.871 s]
[INFO] Sqoop SPI .......................................... SUCCESS [  0.539 s]
[INFO] Sqoop Core ......................................... SUCCESS [  4.263 s]
[INFO] Sqoop Repository ................................... SUCCESS [  0.012 s]
[INFO] Sqoop Common Repository ............................ SUCCESS [  1.226 s]
[INFO] Sqoop Derby Repository ............................. SUCCESS [01:02 min]
[INFO] Sqoop Tools ........................................ SUCCESS [  0.541 s]
[INFO] Sqoop Security ..................................... SUCCESS [  0.499 s]
[INFO] Sqoop Execution Engines ............................ SUCCESS [  0.011 s]
[INFO] Sqoop Mapreduce Execution Engine ................... SUCCESS [ 27.692 s]
[INFO] Sqoop Submission Engines ........................... SUCCESS [  0.011 s]
[INFO] Sqoop Mapreduce Submission Engine .................. SUCCESS [  0.680 s]
[INFO] Sqoop Generic JDBC Connector ....................... SUCCESS [ 15.845 s]
[INFO] Sqoop HDFS Connector ............................... SUCCESS [  9.887 s]
[INFO] Sqoop Kite Connector ............................... SUCCESS [  5.060 s]
[INFO] Sqoop Kafka Connector .............................. SUCCESS [  9.216 s]
[INFO] Sqoop Server ....................................... SUCCESS [  3.281 s]
[INFO] Sqoop Client ....................................... SUCCESS [  1.954 s]
[INFO] Sqoop Shell ........................................ SUCCESS [  1.860 s]
[INFO] Sqoop Documentation ................................ SUCCESS [ 23.742 s]
[INFO] Sqoop Tomcat additions ............................. SUCCESS [  0.458 s]
[INFO] Sqoop Distribution ................................. SUCCESS [  1.770 s]
[INFO] Sqoop Integration Tests ............................ FAILURE [19:14 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 22:30 min
[INFO] Finished at: 2014-12-16T09:00:59-08:00
[INFO] Final Memory: 285M/953M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12:test (integration-test)
on project test: There are test failures.
[ERROR] 
[ERROR] Please refer to /Users/vbasavaraj/Projects/sqoop2/test/target/surefire-reports for
the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following
articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :test
{code}


was (Author: vybs):
[~abec] the last patch has the lingering "updateSubmission" call, hence you saw those errors.

This one does have failures via local runner ( for FromRDBMSToHDFSTest) , but passes all the
cluster based integration tests

{code}
[INFO] Surefire report directory: /Users/vbasavaraj/Projects/sqoop2/test/target/surefire-reports

-------------------------------------------------------
 T E S T S
-------------------------------------------------------

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.sqoop.integration.connector.jdbc.generic.FromHDFSToRDBMSTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.808 sec
Running org.apache.sqoop.integration.connector.jdbc.generic.FromRDBMSToHDFSTest
Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 22.408 sec <<< FAILURE!
Running org.apache.sqoop.integration.connector.jdbc.generic.PartitionerTest
Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 196.276 sec
Running org.apache.sqoop.integration.connector.jdbc.generic.TableStagedRDBMSTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 12.217 sec
Running org.apache.sqoop.integration.connector.kafka.FromRDBMSToKafkaTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.683 sec
Running org.apache.sqoop.integration.repository.derby.upgrade.Derby1_99_4UpgradeTest
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 121.865 sec <<< FAILURE!
Running org.apache.sqoop.integration.server.SubmissionWithDisabledModelObjectsTest
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.176 sec
Running org.apache.sqoop.integration.server.VersionTest
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 8.595 sec <<< FAILURE!

Results :

Failed tests:   testColumns(org.apache.sqoop.integration.connector.jdbc.generic.FromRDBMSToHDFSTest)
  testVersion(org.apache.sqoop.integration.server.VersionTest): expected:<[Mon Dec 15 17:45:39]
PST 2014> but was:<[Sat Dec 13 18:20:35] PST 2014>

Tests in error: 
  testPostUpgrade(org.apache.sqoop.integration.repository.derby.upgrade.Derby1_99_4UpgradeTest):
Failed to start the Tomcat 6.x container. Check the [/Users/vbasavaraj/Projects/sqoop2/test/target/sqoop-cargo-tests/org.apache.sqoop.integration.repository.derby.upgrade.Derby1_99_4UpgradeTest/testPostUpgrade/sqoop-mini-cluster//log/tomcat.log]
file containing the container logs for more details.

Tests run: 30, Failures: 2, Errors: 1, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 07:16 min
[INFO] Finished at: 2014-12-16T08:22:55-08:00
[INFO] Final Memory: 70M/801M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.

{code}

> Sqoop2: Submission Engine does not set all details on SubmissionRecord in Local mode
> ------------------------------------------------------------------------------------
>
>                 Key: SQOOP-1879
>                 URL: https://issues.apache.org/jira/browse/SQOOP-1879
>             Project: Sqoop
>          Issue Type: Sub-task
>            Reporter: Veena Basavaraj
>            Assignee: Veena Basavaraj
>         Attachments: SQOOP-1879-v1.patch, SQOOP-1879-v2.patch, SQOOP-1879-v4.patch, SQOOP-1879-v5.patch
>
>
> {code}
>       // If we're in local mode than wait on completion. Local job runner do not
>       // seems to be exposing API to get previously submitted job which makes
>       // other methods of the submission engine quite useless.
>       if(isLocal()) {
>         job.waitForCompletion(true);
>       } else {
>         job.submit();
>       }
> {code}
> the above code says that the job.waitForCompletion(true) is blocking and hence the other
submisison engine apis are useless in this local mode.
> But we dont set the all attributes of the job run into the submission record in this
case 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message