tika-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Allison (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TIKA-2833) Add a CSV/TSV detector
Date Thu, 28 Feb 2019 19:38:00 GMT

    [ https://issues.apache.org/jira/browse/TIKA-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16780855#comment-16780855

Tim Allison commented on TIKA-2833:

One of the weaknesses of {{tika-eval}} is the focus on alphabetic tokens...so, by that metric
we are now losing a small percentage of "common words" because of the new exceptions.  However,
what we aren't able to detect easily...er...we don't have a report for this...is the decrease
in tokens that look like "10,2,3,3,2,3,232342,23424" etc...

> Add a CSV/TSV detector
> ----------------------
>                 Key: TIKA-2833
>                 URL: https://issues.apache.org/jira/browse/TIKA-2833
>             Project: Tika
>          Issue Type: Task
>            Reporter: Tim Allison
>            Priority: Minor
>         Attachments: csv_reports.zip
> Given initial experimentation, I think we can fairly easily add a fairly robust CSV/TSV
detector that will identify well-formed (ha!) csvs and return the charset encoding and the

This message was sent by Atlassian JIRA

View raw message