nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Markus Jelsma (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (NUTCH-1726) HeadingsFilter does not find nested nodes
Date Thu, 13 Feb 2014 09:49:20 GMT

     [ https://issues.apache.org/jira/browse/NUTCH-1726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Markus Jelsma updated NUTCH-1726:
---------------------------------

    Attachment: NUTCH-1726-trunk.patch

Thanks Lufeng! Here's another patch with additional options. Because it found nested nodes
it suddenly returned a lot of headings for a given URL.

* headings.limit similar to multivalued but allows to limit the number of headings per element
* headings.maxlength max length of heading
* headings.truncate what to do with too long headings, truncate or skip them?
* headings.minlength obvious
* headings.ignore.hyperlinks will ignore headings inside anchors

The headings.ignore.hyperlinks does not work despite the nodewalker.skipChildren() call. Haven't
figured this out yet.

> HeadingsFilter does not find nested nodes
> -----------------------------------------
>
>                 Key: NUTCH-1726
>                 URL: https://issues.apache.org/jira/browse/NUTCH-1726
>             Project: Nutch
>          Issue Type: Bug
>    Affects Versions: 1.7
>            Reporter: Markus Jelsma
>            Assignee: Markus Jelsma
>            Priority: Minor
>             Fix For: 1.8
>
>         Attachments: NUTCH-1726-trunk-v2.patch, NUTCH-1726-trunk.patch, NUTCH-1726-trunk.patch
>
>
> Filter won't find:
> {code}
> <h1><span>apache nutch</span></h1>
> {code}
> The getNodeValue() tries to read data from children but should traverse nodes instead.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message