nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (NUTCH-2139) Basic plugin to index inlinks and outlinks
Date Thu, 15 Oct 2015 18:51:05 GMT

    [ https://issues.apache.org/jira/browse/NUTCH-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14959407#comment-14959407
] 

ASF GitHub Bot commented on NUTCH-2139:
---------------------------------------

Github user sebastian-nagel commented on a diff in the pull request:

    https://github.com/apache/nutch/pull/78#discussion_r42164537
  
    --- Diff: conf/nutch-default.xml ---
    @@ -1896,4 +1896,33 @@ CAUTION: Set the parser.timeout to -1 or a bigger value than 30,
when using this
       </description>
     </property>
     
    +<!-- index-links plugin -->
    +
    +<property>
    +  <name>outlinks.host.ignore</name>
    +  <value>false</value>
    +  <description>
    +    Ignore the outlinks that point out to the same host as the URL being indexed. 
    +    By default all outlinks are indexed.
    +  </description>
    +</property>
    +
    +<property>
    +  <name>inlinks.host.ignore</name>
    +  <value>false</value>
    +  <description>
    +    Ignore the inlinks coming from the same host as the URL being indexed. By default

    +    all inlinks are indexed.
    +  </description>
    --- End diff --
    
    If db.ignore.internal.links == true (that's the default) this option wouldn't change anything
because host-internal-inlinks aren't even stored in linkdb. Could be worth to place a reference
to db.ignore.internal.links.


> Basic plugin to index inlinks and outlinks
> ------------------------------------------
>
>                 Key: NUTCH-2139
>                 URL: https://issues.apache.org/jira/browse/NUTCH-2139
>             Project: Nutch
>          Issue Type: Improvement
>          Components: indexer, plugin
>            Reporter: Jorge Luis Betancourt Gonzalez
>            Priority: Minor
>              Labels: link, plugin
>             Fix For: 1.11
>
>
> Basic plugin that allows to index the inlinks and outlinks of the web pages, this could
be very useful for analytic purposes, including neat visualizations using d3.js for instance.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message