nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Markus Jelsma (JIRA)" <>
Subject [jira] [Commented] (NUTCH-923) Multilingual support for Solr-index-mapping
Date Fri, 03 Aug 2012 10:39:02 GMT


Markus Jelsma commented on NUTCH-923:

If you specify langid.overwrite=false and provide the language in the landid.langField the
LangIdentifier skips detection and only maps the language fields. We use it that way with
a custom language identifier so we can do proper analysis for many languages.
> Multilingual support for Solr-index-mapping
> -------------------------------------------
>                 Key: NUTCH-923
>                 URL:
>             Project: Nutch
>          Issue Type: Improvement
>          Components: indexer
>    Affects Versions: 1.2
>            Reporter: Matthias Agethle
>            Assignee: Markus Jelsma
>            Priority: Minor
>         Attachments: patch-923-nutch-release-1.2.txt
> It would be useful to extend the mapping-possibilites when indexing to solr.
> One useful feature would be to use the detected language of the html page (for example
via the language-identifier plugin) and send the content to corresponding language-aware solr-fields.
> The mapping file could be as follows:
> <field dest="lang" source="lang"/>
> <field dest="title_${lang}" source="title" />
> so that the title-field gets mapped to title_en for English-pages and tilte_fr for French
> What do you think? Could this be useful also to others?
> Or are there already other solutions out there?

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message