nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julien Nioche (JIRA)" <>
Subject [jira] [Created] (NUTCH-1747) Use AtomicInteger as semaphore in Fetcher
Date Sat, 05 Apr 2014 07:27:14 GMT
Julien Nioche created NUTCH-1747:

             Summary: Use AtomicInteger as semaphore in Fetcher
                 Key: NUTCH-1747
             Project: Nutch
          Issue Type: Improvement
          Components: fetcher
    Affects Versions: 1.8
            Reporter: Julien Nioche
            Priority: Minor

In Fetcher we currently use 

Set<FetchItem>  inProgress = Collections.synchronizedSet(new HashSet<FetchItem>());

as semaphore within the FetchItemQueues to keep track of the URLs being fetched and prevent
threads from pulling from them. It works fine but we could use AtomicIntegers instead as all
we need is the counts, not the contents.

This change would have little impact on the performance but would make the code a bit cleaner.

This message was sent by Atlassian JIRA

View raw message