sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Malaska" <ted.mala...@cloudera.com>
Subject Review Request 34038: SQOOP-2151 - Sqoop2: Sqoop mapreduce job gets into deadlock when loader throws an exception
Date Mon, 11 May 2015 13:50:59 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34038/
-----------------------------------------------------------

Review request for Sqoop and Gwen Shapira.


Repository: sqoop-sqoop2


Description
-------

If there are exceptions in the SqoopOutputFormatLoadExecutor around line 261 and then you
call close you can get stuck.


Diffs
-----

  execution/mapreduce/src/main/java/org/apache/sqoop/job/mr/SqoopOutputFormatLoadExecutor.java
fc18586 
  execution/mapreduce/src/test/java/org/apache/sqoop/job/mr/TestSqoopOutputFormatLoadExecutor.java
71c98db 

Diff: https://reviews.apache.org/r/34038/diff/


Testing
-------

If you add the following into line 261 to SqoopOutputFormatLoadExecutor: 

int i = 0;
i = i/i;

You will get stuck if you have a try catch block around the writing and you call the close
at the end.

Here is the events in the two threads that make this happen.

How I got it stuck
- Thread 1 - Lock free
- Thread 2 - Exception
- Thread 2 - Released Free
- Thread 1 - Lock free
- Thread 1 - future.get
- Thread 1 - Exception thrown
- Thread 1 - Exception catched
- Thread 1 - close
- Thread 1 - Try to lock free
- Stuck


Thanks,

Ted Malaska


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message