nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sujen Shah (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (NUTCH-2015) Make FetchNodeDb optional (off by default) if NutchServer is not used
Date Fri, 29 May 2015 23:50:17 GMT

    [ https://issues.apache.org/jira/browse/NUTCH-2015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14565668#comment-14565668
] 

Sujen Shah commented on NUTCH-2015:
-----------------------------------

Hi [~wastl-nagel], 
I updated the code as you suggested. Have put the parsing and the server state check outside
the loop and created new FetchNodes only if both conditions are true. 

> Make FetchNodeDb optional (off by default) if NutchServer is not used
> ---------------------------------------------------------------------
>
>                 Key: NUTCH-2015
>                 URL: https://issues.apache.org/jira/browse/NUTCH-2015
>             Project: Nutch
>          Issue Type: Sub-task
>          Components: fetcher, REST_api
>            Reporter: Sujen Shah
>            Assignee: Chris A. Mattmann
>              Labels: memex
>             Fix For: 1.11
>
>
> Currently, the FetchNodes are created even if the NutchServer is not used causing memory
exceptions. This patch makes the fetcher report to the FetchNodeDb only if the crawl is invoked
from the REST service (ie NutchServer)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message