sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fero Szabo via Review Board <nore...@reviews.apache.org>
Subject Re: Review Request 67675: SQOOP-3332 Extend Documentation of --resilient flag and add warning message when detected
Date Mon, 25 Jun 2018 15:17:05 GMT

This is an automatically generated e-mail. To reply, visit:

(Updated June 25, 2018, 3:17 p.m.)

Review request for Sqoop, Boglarka Egyed, daniel voros, and Szabolcs Vasas.


Added a fix for a bug in SQLServerManagerImportTest, introduced in SQOOP-3333: while the constructor
of the testclass is invoked for every test again and a again, the configuration object are
only instatiated once. Thus 5 tests were reusing the same configuration references, polluting
the builders. 
(in the end the builder contained the following tool options: --table-hints NOLOCK --table-hints
NOLOCK,NOWAIT --non-resilient --resilient)

This fix avoids that by creating a new copy of the builder, each time the constructor is invoked

Bugs: SQOOP-3332

Repository: sqoop-trunk


This is the documentation part of SQOOP-3333.

Diffs (updated)

  src/docs/user/connectors.txt f1c7aebe 
  src/java/org/apache/sqoop/manager/SQLServerManager.java c98ad2db 
  src/java/org/apache/sqoop/manager/SqlServerManagerContextConfigurator.java cf58f631 
  src/test/org/apache/sqoop/manager/sqlserver/SQLServerManagerImportTest.java fc1c4895 

Diff: https://reviews.apache.org/r/67675/diff/2/

Changes: https://reviews.apache.org/r/67675/diff/1-2/


Unit tests, 3rdparty tests, ant docs.

I've also investigated how export and import works: 

Import has it's retry mechanism in org.apache.sqoop.mapreduce.db.SQLServerDBRecordReader#nextKeyValue
In case of error, it re-calculates the db query, thus the implicit requirements

Export has it's retry loop in org.apache.sqoop.mapreduce.SQLServerAsyncDBExecThread#write
It doesn't recalculate the query, thus is a lot safer.


Fero Szabo

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message