nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Markus Jelsma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (NUTCH-2014) Fetcher hang-up on completion
Date Mon, 18 May 2015 22:05:00 GMT

    [ https://issues.apache.org/jira/browse/NUTCH-2014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14549335#comment-14549335
] 

Markus Jelsma commented on NUTCH-2014:
--------------------------------------

Ok, please ignore. I was fooled by Lewis' message on 1934 at 21st april. This does not appear
to be in 1.10. Great work :)

> Fetcher hang-up on completion
> -----------------------------
>
>                 Key: NUTCH-2014
>                 URL: https://issues.apache.org/jira/browse/NUTCH-2014
>             Project: Nutch
>          Issue Type: Bug
>          Components: fetcher
>    Affects Versions: 1.11
>            Reporter: Sebastian Nagel
>            Assignee: Sebastian Nagel
>            Priority: Critical
>             Fix For: 1.11
>
>         Attachments: NUTCH-2014-v1.patch
>
>
> Although fetcher has done its work it does not shut down and exit but continues to log
(and before reports its status to the task tracker):
> {noformat}
> -activeThreads=11, spinWaiting=0, fetchQueues.totalSize=33, fetchQueues.getQueueCount=1
> -activeThreads=11, spinWaiting=10, fetchQueues.totalSize=26, fetchQueues.getQueueCount=1
> -activeThreads=11, spinWaiting=9, fetchQueues.totalSize=9, fetchQueues.getQueueCount=1
> -activeThreads=9, spinWaiting=7, fetchQueues.totalSize=0, fetchQueues.getQueueCount=1
> -activeThreads=1, spinWaiting=0, fetchQueues.totalSize=0, fetchQueues.getQueueCount=0
> -activeThreads=1, spinWaiting=0, fetchQueues.totalSize=0, fetchQueues.getQueueCount=0
> ...
> (last message continues)
> {noformat}
> A possible hint: activeThreads should never exceed 10 (configured per default). Looks
like the corresponding variable is lost/mixed-up during fetcher refactorization (NUTCH-1934).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message