nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack@archive.org (JIRA)" <j...@apache.org>
Subject [jira] Commented: (NUTCH-426) parse-js skips parsing if found URL fails java.net.URL parse
Date Thu, 04 Jan 2007 20:14:27 GMT

    [ https://issues.apache.org/jira/browse/NUTCH-426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12462307
] 

stack@archive.org commented on NUTCH-426:
-----------------------------------------

Just attached a patch that catches the MalformedURLException, logs the failure at trace level,
and then continues to pickup on any more matches found in remaining javascript.

> parse-js skips parsing if found URL fails java.net.URL parse
> ------------------------------------------------------------
>
>                 Key: NUTCH-426
>                 URL: https://issues.apache.org/jira/browse/NUTCH-426
>             Project: Nutch
>          Issue Type: Bug
>          Components: fetcher
>    Affects Versions: 0.9.0
>            Reporter: stack@archive.org
>            Priority: Minor
>         Attachments: nutch426.patch
>
>
> The parse-js plugin in getJSLinks tries a regex looking for likely URLs against a string
of javascript.  Any matches that do not begin 'www' are given to java.net.URL with base URL
to test 'URLness'.  Often this test will fail.  Currently, when it fails, nutch skips processing
any more of the javascript snippet logs an error.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message