nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrzej Bialecki>
Subject Re: problems http-client
Date Mon, 19 Dec 2005 18:47:17 GMT
Stefan Groschupf wrote:

> Anyway today we note that when fetching with http-client the sum of  
> errors and fetched pages is  much less than the size defined when  
> generating the segment.
> Changing to protocol-http solves the problem.
> Has anyone also note this behavior?

I haven't, but this plugin is known to have some issues... Could you add 
some log messages here and there to confirm this, like counting the 
number of invocations of getProtocolOutput in protocol-httpclient vs. 
the number of calls to FetcherThread.output(). This could be a bug 
somewhere in the redirect handling code.

Best regards,
Andrzej Bialecki     <><
 ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration  Contact: info at sigram dot com

View raw message