sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Szabolcs Vasas <vasas.szabo...@gmail.com>
Subject Re: Review Request 68606: Error during direct Netezza import/export can interrupt process in uncontrolled ways
Date Wed, 05 Sep 2018 05:05:55 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68606/#review208341
-----------------------------------------------------------



Hi Daniel,

Thank you for submitting this patch, I think it is a good idea to use AtomicBoolean instead
of the interrupted flag.
I have left a small question only.
The unit and third party tests are running fine.


src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableExportMapper.java
Line 229 (original)
<https://reviews.apache.org/r/68606/#comment292196>

    The isAlive check is not removed in NetezzaExternalTableImportMapper, is there a particular
reason for that?


- Szabolcs Vasas


On Sept. 3, 2018, 11:32 a.m., daniel voros wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68606/
> -----------------------------------------------------------
> 
> (Updated Sept. 3, 2018, 11:32 a.m.)
> 
> 
> Review request for Sqoop.
> 
> 
> Bugs: SQOOP-3378
>     https://issues.apache.org/jira/browse/SQOOP-3378
> 
> 
> Repository: sqoop-trunk
> 
> 
> Description
> -------
> 
> `SQLException` during JDBC operation in direct Netezza import/export signals parent thread
to fail fast by interrupting it.
> We're trying to process the interrupt in the parent (main) thread, but there's no guarantee
that we're not in some internal call that will process the interrupted flag and reset it before
we're able to check.
> 
> It is also possible that the parent thread has passed the "checking part" when it gets
interrupted. In case of `NetezzaExternalTableExportMapper` this can interrupt the upload of
log files.
> 
> I'd recommend using some other means of communication between the threads than interrupts.
> 
> 
> Diffs
> -----
> 
>   src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableExportMapper.java
5bf21880 
>   src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaExternalTableImportMapper.java
306062aa 
>   src/java/org/apache/sqoop/mapreduce/db/netezza/NetezzaJDBCStatementRunner.java cedfd235

>   src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableExportMapper.java
PRE-CREATION 
>   src/test/org/apache/sqoop/mapreduce/db/netezza/TestNetezzaExternalTableImportMapper.java
PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/68606/diff/1/
> 
> 
> Testing
> -------
> 
> added new UTs and checked manual Netezza tests (NetezzaExportManualTest, NetezzaImportManualTest)
> 
> 
> Thanks,
> 
> daniel voros
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message