nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steven Denny (JIRA)" <j...@apache.org>
Subject [jira] Commented: (NUTCH-719) fetchQueues.totalSize incorrect in Fetcher2
Date Fri, 10 Jul 2009 16:13:14 GMT

    [ https://issues.apache.org/jira/browse/NUTCH-719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12729699#action_12729699
] 

Steven Denny commented on NUTCH-719:
------------------------------------

I've changed line 324 of src/java/org/apache/nutch/fetcher/Fetcher.java to 

public void synchronized void addFetchItem(FetchItem it) {

(added the synchronized) and initial testing looks good.



> fetchQueues.totalSize incorrect in Fetcher2
> -------------------------------------------
>
>                 Key: NUTCH-719
>                 URL: https://issues.apache.org/jira/browse/NUTCH-719
>             Project: Nutch
>          Issue Type: Bug
>          Components: fetcher
>    Affects Versions: 1.0.0
>            Reporter: Julien Nioche
>
> I had a look at the logs generated by Fetcher2 and found cases where there were no active
fetchQueues but fetchQueues.totalSize was != 0
> fetcher.Fetcher2 - -activeThreads=200, spinWaiting=200, fetchQueues.totalSize=1, fetchQueues=0
> since the code relies on fetchQueues.totalSize to determine whether the work is finished
or not the task is blocked until the abortion mechanism kicks in
> 2009-03-12 09:27:38,977 WARN  fetcher.Fetcher2 - Aborting with 200 hung threads.
> could that be a synchronisation issue? any ideas?

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message