[ https://issues.apache.org/jira/browse/NUTCH-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15964262#comment-15964262
]
Markus Jelsma commented on NUTCH-1932:
--------------------------------------
We have this latest patch running for over six months now and i haven't seen any oddities
yet. Stuff checks out, records are first removed from the index, later from the crawldb. Any
final comments on this issue?
> Automatically remove orphaned pages
> -----------------------------------
>
> Key: NUTCH-1932
> URL: https://issues.apache.org/jira/browse/NUTCH-1932
> Project: Nutch
> Issue Type: New Feature
> Reporter: Markus Jelsma
> Assignee: Markus Jelsma
> Priority: Minor
> Attachments: NUTCH-1932-add.patch, NUTCH-1932.patch, NUTCH-1932.patch, NUTCH-1932.patch,
NUTCH-1932.patch, NUTCH-1932.patch, NUTCH-1932.patch, NUTCH-1932.patch, NUTCH-1932.patch,
NUTCH-1932.patch, NUTCH-1932.patch, NUTCH-1932.patch, NUTCH-1932.patch, NUTCH-1932.patch,
NUTCH-1932.patch
>
>
> Orphan scoring filter that determines whether a page has become orphaned, e.g. it has
no more other pages linking to it. If a page hasn't been linked to after markGoneAfter seconds,
the page is marked as gone and is then removed by an indexer. If a page hasn't been linked
to after markOrphanAfter seconds, the page is removed from the CrawlDB.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
|