tika-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tim Allison (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TIKA-1599) Switch from TagSoup to JSoup
Date Wed, 09 Dec 2015 17:06:11 GMT

    [ https://issues.apache.org/jira/browse/TIKA-1599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15048978#comment-15048978
] 

Tim Allison commented on TIKA-1599:
-----------------------------------

bq. So I'm not seeing how the parser that CommonCrawl uses would factor into that.

Right, agree.  I was just seeing if they had also picked up JSoup or were using something
else.

bq. But I also don't have a good suggestion for how to automatically evaluate the TagSoup
vs. JSoup parse-off, other than simple measures like how many failed, or maybe the amount
of text extracted?

Y, agreed.  For PDFBOX-3058, and as part of TIKA-1330, I have some alpha-level eval code that
will do these comparisons.  Following [~tilman]'s recommendation, I added counts for "common
English" words.

> Switch from TagSoup to JSoup
> ----------------------------
>
>                 Key: TIKA-1599
>                 URL: https://issues.apache.org/jira/browse/TIKA-1599
>             Project: Tika
>          Issue Type: Improvement
>          Components: parser
>    Affects Versions: 1.7, 1.8
>            Reporter: Ken Krugler
>            Assignee: Ken Krugler
>            Priority: Minor
>
> There are several Tika issues related to how TagSoup cleans up HTML ([TIKA-381], [TIKA-985],
maybe [TIKA-715]), but TagSoup doesn't seem to be under active development.
> On the other hand I know of several projects that are now using [JSoup|https://github.com/jhy/jsoup],
which is an active project (albeit only one main contributor) under the MIT license.
> I haven't looked into how hard it would be to switch this dependency.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message