nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Markus Jelsma (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (NUTCH-79) Fault tolerant searching.
Date Fri, 01 Apr 2011 14:29:05 GMT

     [ https://issues.apache.org/jira/browse/NUTCH-79?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Markus Jelsma closed NUTCH-79.
------------------------------

    Resolution: Won't Fix

Bulk close of legacy issues:
http://www.lucidimagination.com/search/document/2738eeb014805854/clean_up_open_legacy_issues_in_jira

> Fault tolerant searching.
> -------------------------
>
>                 Key: NUTCH-79
>                 URL: https://issues.apache.org/jira/browse/NUTCH-79
>             Project: Nutch
>          Issue Type: New Feature
>          Components: searcher
>            Reporter: Piotr Kosiorowski
>         Attachments: patch
>
>
> I have finally managed to prepare first version of fault tolerant searching I have promised
long time ago. 
> It reads server configuration from search-groups.txt file (in startup directory or directory
specified by searcher.dir) if no search-servers.txt file is present. If search-servers.txt
 is presentit would be read and handled as previously.
> ---------------------------------------------------
> Format of search-groups.txt:
> * <pre>
>  *  search.group.count=[int] 
>  *  search.group.name.[i]=[string] (for i=0 to count-1)
>  *  
>  *  For each name: 
>  *  [name].part.count=[int] partitionCount 
>  *  [name].part.[i].host=[string] (for i=0 to partitionCount-1)
>  *  [name].part.[i].port=int (for i=0 to partitionCount-1)
>  *  
>  *  Example: 
>  *  search.group.count=2 
>  *  search.group.name.0=master
>  *  search.group.name.1=backup
>  *  
>  *  master.part.count=2 
>  *  master.part.0.host=host1 
>  *  master.part.0.port=7777
>  *  master.part.1.host=host2 
>  *  master.part.1.port=7777
>  *  
>  *  backup.part.count=2 
>  *  backup.part.0.host=host3 
>  *  backup.part.0.port=7777
>  *  backup.part.1.host=host4 
>  *  backup.part.1.port=7777
>  * </pre>.
> ------------------------------------------------
> If more than one search group is defined in configuration file requests are distributed
among groups in round-robin fashion. If one of the servers from the group fails to respond
the whole group is treated as inactive and removed from the pool used to distributed requests.
There is a separate recovery thread that every "searcher.recovery.delay" seconds (default
60) tries to check if inactive became alive and if so adds it back to the pool of active groups.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message