sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From daniel voros <daniel.vo...@gmail.com>
Subject Review Request 68606: Error during direct Netezza import/export can interrupt process in uncontrolled ways
Date Mon, 03 Sep 2018 11:32:36 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68606/
-----------------------------------------------------------

Review request for Sqoop.


Bugs: SQOOP-3378
    https://issues.apache.org/jira/browse/SQOOP-3378


Repository: sqoop-trunk


Description
-------

`SQLException` during JDBC operation in direct Netezza import/export signals parent thread
to fail fast by interrupting it.
We're trying to process the interrupt in the parent (main) thread, but there's no guarantee
that we're not in some internal call that will process the interrupted flag and reset it before
we're able to check.

It is also possible that the parent thread has passed the "checking part" when it gets interrupted.
In case of `NetezzaExternalTableExportMapper` this can interrupt the upload of log files.

I'd recommend using some other means of communication between the threads than interrupts.


Diffs
-----

  src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableExportMapper.java 5bf21880

  src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableImportMapper.java 306062aa

  src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaJDBCStatementRunner.java cedfd235

  src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableExportMapper.java
PRE-CREATION 
  src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableImportMapper.java
PRE-CREATION 


Diff: https://reviews.apache.org/r/68606/diff/1/


Testing
-------

added new UTs and checked manual Netezza tests (NetezzaExportManualTest, NetezzaImportManualTest)


Thanks,

daniel voros


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message