nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lewis John McGibbney (JIRA)" <>
Subject [jira] [Commented] (NUTCH-2038) url filter that uses a model (from a classifier)
Date Tue, 16 Jun 2015 18:00:00 GMT


Lewis John McGibbney commented on NUTCH-2038:

I just spoke with [~sujenshah], offline, he is adapting his scoring-similarity plugin in NUTCH-2039
to offer a ScoringSimilarity interface which could easily be utilized here. I am not convinced
that this is a URLFilter due to the fact that URLFilter work on lists of URLs as defined by
the URLFilter#filter(String urlString) interface.
I continue to offer my opinion that model-based classification has the purpose of offering
a scoring metric and should hence be implemented as a scoring plugin... the ongoing work being
undertaken by [~sujenshah] facilitates your work [~asitang] but implementing ScoringFilter
as oppose to URLFilter. 

> url filter that uses a model (from a classifier) 
> -------------------------------------------------
>                 Key: NUTCH-2038
>                 URL:
>             Project: Nutch
>          Issue Type: New Feature
>          Components: fetcher, injector, parser
>            Reporter: Asitang Mishra
>            Assignee: Chris A. Mattmann
>              Labels: memex, nutch
>             Fix For: 1.11
> A url filter that will filter out the urls (after the parsing stage,  will keep only
those urls that contain some "hot words" provided again in a list.) from that pages that are
classified irrelevant by the classifier (using a model provided).

This message was sent by Atlassian JIRA

View raw message