nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Markus Jelsma (Commented) (JIRA)" <>
Subject [jira] [Commented] (NUTCH-1194) CrawlDB lock should be released earlier
Date Fri, 04 Nov 2011 13:21:00 GMT


Markus Jelsma commented on NUTCH-1194:

The comment above mine was removed by the user who was writing about "locking should be done
in setup/cleanup task.". This CrawlDB lock spans multiple mapred jobs which makes the difference.
> CrawlDB lock should be released earlier
> ---------------------------------------
>                 Key: NUTCH-1194
>                 URL:
>             Project: Nutch
>          Issue Type: Improvement
>          Components: generator
>            Reporter: Markus Jelsma
>            Assignee: Markus Jelsma
>            Priority: Minor
>             Fix For: 1.5
> Lock on the CrawlDB is released when everything is finished. But when generating many
segments, the lock remains in place while it's not neccessary anymore. If GENERATE_UPDATE_DB
is false we can release the lock immediately after the selector has finished.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message