lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lance Norskog (JIRA)" <>
Subject [jira] [Commented] (SOLR-1979) Create LanguageIdentifierUpdateProcessor
Date Fri, 09 Sep 2011 22:46:09 GMT


Lance Norskog commented on SOLR-1979:

I'm impressed! This is a lot of work and empirical testing for a difficult problem.

There are a few parameters that are true/false, but in the future you might want a third answer.
It might be worth making the decision via a keyword so you can add new keywords later.

About the multiple languages in one field problem: you can't solve everything at once. The
other document analysis components like UIMA should be able to identify parts of documents,
and then you use this on one part at a time. This is the point of a modular toolkit: you combine
the tools to solve advanced problems.

> Create LanguageIdentifierUpdateProcessor
> ----------------------------------------
>                 Key: SOLR-1979
>                 URL:
>             Project: Solr
>          Issue Type: New Feature
>          Components: update
>            Reporter: Jan Høydahl
>            Assignee: Jan Høydahl
>            Priority: Minor
>              Labels: UpdateProcessor
>             Fix For: 3.5
>         Attachments: SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch,
SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch
> Language identification from document fields, and mapping of field names to language-specific
fields based on detected language.
> Wrap the Tika LanguageIdentifier in an UpdateProcessor.

This message is automatically generated by JIRA.
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message