nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sherjeel Niazi <sherj...@softmatics.com>
Subject How to crawl every URL of website
Date Fri, 17 Apr 2009 07:42:49 GMT
Hello there,

I would like to know that if I have to crawl every URL of a web site (I
myself do not know how many URL's are there).
Website structure is like:
   www.domain.com/record/1
   www.domain.com/record/2
   www.domain.com/record/3
   ......................
   so on



Thanks,
Sherjeel

Mime
View raw message