tika-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Allison (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (TIKA-1001) tika no longer seems to honor HTTP meta tag for arabic text in ISO-8859-6 charset
Date Mon, 12 Aug 2013 14:22:53 GMT

     [ https://issues.apache.org/jira/browse/TIKA-1001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Tim Allison updated TIKA-1001:

    Attachment: TIKA-1001v1.tar.gz

This is a draft that simplifies the extraction of the charset attribute within a <meta>
tag (old html and new HTML5) and should make the charset extraction more robust to noisy metaheaders.

The strategy is:
1) find the <meta> tags
2) find charset=x within the meta tag
3) return the first valid charset

Is the proposed strategy too broad?  Will there be false positives?

Will commit in a few days if there is no feedback.  Thank you!

P.S. Ignore the patch.xml file, of course. :)
> tika no longer seems to honor HTTP meta tag for arabic text in ISO-8859-6 charset
> ---------------------------------------------------------------------------------
>                 Key: TIKA-1001
>                 URL: https://issues.apache.org/jira/browse/TIKA-1001
>             Project: Tika
>          Issue Type: Bug
>          Components: parser
>    Affects Versions: 1.2
>            Reporter: david lemon
>         Attachments: badarabic.html, TIKA-1001v1.tar.gz
> attached document extracts correctly in Tika 1.1
> attached document extracts incorrectly in tika 1.2.
> The difference appears to be that tika 1.1 honors the http meta content-type tag which
specifies the charset as iso-8859-6, and correctly converts the output to UTF-8.
> tika 1.2 appears to ignore the charset specified in the meta tag.
> Some noodling seems to indicate that the problem is the charset.
> it doesn't matter what mode tika is used in (server, app mode, etc. even if content-type
is specified with a charset, the output is still garbage).

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message