nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Markus Jelsma <markus.jel...@openindex.io>
Subject Re: exposing generator.max.num.segments in nutch-default.xml and to Crawl command
Date Tue, 06 Sep 2011 15:13:06 GMT
Yes it is possible but i don't believe it's something you'd want to as there's 
an issue to deprecate the crawl class and replace it with a shell script 
example.

https://issues.apache.org/jira/browse/NUTCH-1087
See the thread as well.

You can easily read configuration settings via jobconf and set the appropriate 
value for the generator in the Crawl class.


> Currently (Nutch 1.3) generator.max.num.segments has no effect when
> passed on to Crawl command. (It is always set to 1). However when using
> Generator in an explicit call it is possible to use custom number of
> segments.
> 
> Is it possibly to read conf for generator.max.num.segments for a Crawl
> command and perhaps expose this property to nutch-default.xml? If you
> agree, I will create an issue and supply a patch.

Mime
View raw message