nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Emmanuel Joke (JIRA)" <j...@apache.org>
Subject [jira] Updated: (NUTCH-615) Redirected URL are fetched wihtout setting any FetchInterval
Date Tue, 26 Feb 2008 14:30:56 GMT

     [ https://issues.apache.org/jira/browse/NUTCH-615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Emmanuel Joke updated NUTCH-615:
--------------------------------

    Attachment: NUTCH-615.patch

> Redirected URL are fetched wihtout setting any FetchInterval
> ------------------------------------------------------------
>
>                 Key: NUTCH-615
>                 URL: https://issues.apache.org/jira/browse/NUTCH-615
>             Project: Nutch
>          Issue Type: Bug
>          Components: fetcher
>    Affects Versions: 1.0.0
>            Reporter: Emmanuel Joke
>             Fix For: 1.0.0
>
>         Attachments: NUTCH-615.patch
>
>
> An url which is redirected result to a new URL. We create a new CrawlDatum for the new
URL within the Fetcher but the FetchInterval was not initialized.
> The new url was recorded in the DB with a FetchInterval = 0 and the FetchTime is never
correctly updated to be fetch later in the future. Thus we keep crawling those URL at each
generation.
> This patch fix this issue.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message