sqoop-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SQOOP-2343) AsyncSqlRecordWriter stucks if any exception is thrown out in its close method
Date Fri, 01 May 2015 02:16:07 GMT

    [ https://issues.apache.org/jira/browse/SQOOP-2343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14522668#comment-14522668
] 

Hudson commented on SQOOP-2343:
-------------------------------

FAILURE: Integrated in Sqoop-ant-jdk-1.6-hadoop23 #1197 (See [https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/1197/])
SQOOP-2343: AsyncSqlRecordWriter stucks if any exception is thrown out in its close method
(jarcec: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=89366b49b3d7227180b35726c5c7919a7c94e736)
* src/java/org/apache/sqoop/mapreduce/AsyncSqlRecordWriter.java


> AsyncSqlRecordWriter stucks if any exception is thrown out in its close method
> ------------------------------------------------------------------------------
>
>                 Key: SQOOP-2343
>                 URL: https://issues.apache.org/jira/browse/SQOOP-2343
>             Project: Sqoop
>          Issue Type: Bug
>          Components: connectors
>    Affects Versions: 1.4.5
>            Reporter: Yibing Shi
>            Assignee: Yibing Shi
>             Fix For: 1.4.7
>
>         Attachments: SQOOP-2343.patch
>
>
> In class {{AsyncSqlRecordWriter}}, if any exception is thrown in its close method, the
Hadoop MapTask will call this close method once more in case it hasn't been closed. Please
see below code snippet (in method runNewMapper):
> {code}
>     try {
>       input.initialize(split, mapperContext);
>       mapper.run(mapperContext);
>       mapPhase.complete();
>       setPhase(TaskStatus.Phase.SORT);
>       statusUpdate(umbilical);
>       input.close();
>       input = null;
>       output.close(mapperContext);
>       output = null;
>     } finally {
>       closeQuietly(input);
>       closeQuietly(output, mapperContext);
>     }
> {code}
> The second time the close method is called, the main thread will stuck in executeUpdate
when trying to put a new dbOp into the synchronous queue, because at this moment the worker
thread has ended and thus not receiver will take that object, which makes the putter (main
thread) stuck.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message