nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Sparks (JIRA)" <j...@apache.org>
Subject [jira] Created: (NUTCH-690) bug in DomContentUtils.shouldThrowAwayLink?
Date Tue, 17 Feb 2009 21:08:59 GMT
bug in DomContentUtils.shouldThrowAwayLink?
-------------------------------------------

                 Key: NUTCH-690
                 URL: https://issues.apache.org/jira/browse/NUTCH-690
             Project: Nutch
          Issue Type: Bug
    Affects Versions: 0.9.0
            Reporter: Peter Sparks
            Priority: Minor


I found a potential bug in DomContentUtils.shouldThrowAwayLink. It returns true for the 5
links at the top of the home page for www.aksteel.com. Here are the links in the source:
            <a id="Search" href="/search/default.aspx"></a>
            <a id="Investor" style="height: 15px;" href="/investor_information/"></a>
            <a id="Markets" href="/markets_products/"></a>
            <a id="Production" href="/production_facilities/"></a>
            <a id="News" href="/news/"></a>
Perhaps I am just ignorant of what this function is supposed to do but returning true for
these 5 links on that site make that site impossible to crawl.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message