nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sebastian Nagel (JIRA)" <j...@apache.org>
Subject [jira] [Created] (NUTCH-2012) Merge parsechecker and indexchecker
Date Fri, 15 May 2015 13:51:00 GMT
Sebastian Nagel created NUTCH-2012:
--------------------------------------

             Summary: Merge parsechecker and indexchecker
                 Key: NUTCH-2012
                 URL: https://issues.apache.org/jira/browse/NUTCH-2012
             Project: Nutch
          Issue Type: Improvement
    Affects Versions: 1.10
            Reporter: Sebastian Nagel
            Priority: Minor
             Fix For: 1.11


ParserChecker and IndexingFiltersChecker have evolved from simple tools to check parsers and
parsefilters resp. indexing filters to powerful tools which emulate the crawling of a single
URL/document:
- check robots.txt (NUTCH-2002)
- follow redirects (NUTCH-2004)

Keeping both tools in sync takes extra work (cf. NUTCH-1757/NUTCH-2006, also NUTCH-2002, NUTCH-2004
are done only for parsechecker). It's time to merge them
* either into one general debugging tool, keeping parsechecker and indexchecker as aliases
* centralize common code in one utility class



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message