lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lance Norskog (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-1979) Create LanguageIdentifierUpdateProcessor
Date Fri, 09 Sep 2011 22:46:09 GMT

    [ https://issues.apache.org/jira/browse/SOLR-1979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13101612#comment-13101612
] 

Lance Norskog commented on SOLR-1979:
-------------------------------------

I'm impressed! This is a lot of work and empirical testing for a difficult problem.

Comments:
There are a few parameters that are true/false, but in the future you might want a third answer.
It might be worth making the decision via a keyword so you can add new keywords later.

About the multiple languages in one field problem: you can't solve everything at once. The
other document analysis components like UIMA should be able to identify parts of documents,
and then you use this on one part at a time. This is the point of a modular toolkit: you combine
the tools to solve advanced problems.

> Create LanguageIdentifierUpdateProcessor
> ----------------------------------------
>
>                 Key: SOLR-1979
>                 URL: https://issues.apache.org/jira/browse/SOLR-1979
>             Project: Solr
>          Issue Type: New Feature
>          Components: update
>            Reporter: Jan Høydahl
>            Assignee: Jan Høydahl
>            Priority: Minor
>              Labels: UpdateProcessor
>             Fix For: 3.5
>
>         Attachments: SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch,
SOLR-1979.patch, SOLR-1979.patch, SOLR-1979.patch
>
>
> Language identification from document fields, and mapping of field names to language-specific
fields based on detected language.
> Wrap the Tika LanguageIdentifier in an UpdateProcessor.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message