nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sebastian Nagel (JIRA)" <>
Subject [jira] [Created] (NUTCH-1419) parsechecker and indexchecker to report protocol status
Date Tue, 03 Jul 2012 12:47:20 GMT
Sebastian Nagel created NUTCH-1419:

             Summary: parsechecker and indexchecker to report protocol status
                 Key: NUTCH-1419
             Project: Nutch
          Issue Type: Improvement
          Components: indexer, parser
    Affects Versions: nutchgora, 1.6
            Reporter: Sebastian Nagel
            Priority: Minor

Parsechecker and indexchecker should report the protocol status when the fetch was not successful
(status other than 200/ok).

In case of a redirect, the protocol status contains the URL a redirect points to. Usually,
this URL should be checked instead of the original one which is not indexed. The content of
a redirect response is less useful (and often empty):
% nutch indexchecker
contentType: text/html
content :       301 Moved Permanently Moved Permanently The document has moved here . Apache/2.4.1
(Unix) OpenSSL/1.
title : 301 Moved Permanently
host :
tstamp :        Tue Jul 03 13:27:32 CEST 2012
url :

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message