nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <>
Subject Re: Urlfilter Patch
Date Thu, 01 Dec 2005 21:21:14 GMT
Chris Mattmann wrote:
>   In principle, the mimeType system should give us some guidance on
> determining the appropriate mimeType for the content, regardless of whether
> it ends in .foo, .bar or the like.

Right, but the URL filters run long before we know the mime type, in 
order to try to keep us from fetching lots of stuff we can't process. 
The mime type is not known until we've fetched it.


View raw message