manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 小林 茂樹(情報システム本部 / サービス企画部) <shigeki.kobayas...@g.softbank.co.jp>
Subject [ManifoldCF 0.5] The web crawler remains running after a network connection refused
Date Wed, 09 May 2012 02:04:24 GMT
Hi guys.



I need some advice for stopping the MCF web crawler from a running state
when a network connection refused.



I use MCF 0.5 with Solr 3.5. I was testing what would happen to the web
crawler when shutting down the web site that is to be crawled. I checked
the simple history and saw “Connection refused” with status code of “-1”,
that looked fine. But as I was waiting, the job status never changed and
remained running. The crawler never crawls in this situation, but when I
opened the web site, the crawler never started crawling again either.

At least, somehow, I want the crawler to stop from running when a network
connection refused, but I don’t know how. Does anyone have any ideas?

Mime
View raw message