nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris A. Mattmann (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (NUTCH-1905) Nutch index tool should be resilient to segments that don't have crawl_* data
Date Fri, 10 Apr 2015 23:34:12 GMT

     [ https://issues.apache.org/jira/browse/NUTCH-1905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chris A. Mattmann resolved NUTCH-1905.
--------------------------------------
    Resolution: Duplicate

Fixed in NUTCH-1771

> Nutch index tool should be resilient to segments that don't have crawl_* data
> -----------------------------------------------------------------------------
>
>                 Key: NUTCH-1905
>                 URL: https://issues.apache.org/jira/browse/NUTCH-1905
>             Project: Nutch
>          Issue Type: Bug
>          Components: indexer
>            Reporter: Chris A. Mattmann
>            Assignee: Chris A. Mattmann
>             Fix For: 1.11
>
>
> When running the ./bin/nutch index command with the -dir <path/to/segment/dir>
I noticed that if you have a segment directory that doesn't include crawl_* or parse_* data,
that the indexer fails (correctly). However, the indexer should be more resilient in those
cases - we can add a simple check to see if those dirs are present in the segment, and proceed
if they are, otherwise, ignore that segment and print a message (and go to the other segments).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message