tika-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TIKA-2725) Make tika-server robust against ooms/infinite loops/memory leaks
Date Tue, 11 Sep 2018 20:56:00 GMT

    [ https://issues.apache.org/jira/browse/TIKA-2725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16611235#comment-16611235
] 

Hudson commented on TIKA-2725:
------------------------------

FAILURE: Integrated in Jenkins build tika-branch-1x #84 (See [https://builds.apache.org/job/tika-branch-1x/84/])
TIKA-2725 -- checkpoint commit ... basic child process is started...need (tallison: [https://github.com/apache/tika/commit/58dadac9131184e74c468771244595f7daee36c8])
* (add) tika-server/src/test/java/org/apache/tika/server/ServerStatusTest.java
* (edit) tika-server/src/main/java/org/apache/tika/server/TikaServerCli.java
* (add) tika-server/src/test/java/org/apache/tika/server/ServerIntegrationTest.java
* (add) tika-server/src/main/java/org/apache/tika/server/ServerStatus.java
* (add) tika-server/src/main/java/org/apache/tika/server/ServerStatusWatcher.java
* (add) tika-server/src/main/java/org/apache/tika/server/TaskStatus.java
* (add) tika-server/src/main/java/org/apache/tika/server/FileCountExceededException.java
TIKA-2725 -- first working draft...ready for commit and future cleanups (tallison: [https://github.com/apache/tika/commit/e7cef357c662a5a52d84c5254c633b9679bf5ff0])
* (edit) tika-server/src/main/java/org/apache/tika/server/ServerStatusWatcher.java
* (edit) tika-server/src/main/java/org/apache/tika/server/TikaServerCli.java
* (add) tika-server/src/test/resources/mock/real_oom.xml
* (edit) tika-server/src/main/java/org/apache/tika/server/resource/DetectorResource.java
* (edit) tika-server/src/main/java/org/apache/tika/server/ServerStatus.java
* (edit) tika-server/src/test/java/org/apache/tika/server/TikaWelcomeTest.java
* (delete) tika-server/src/test/java/org/apache/tika/server/ServerIntegrationTest.java
* (add) tika-server/src/test/resources/mock/thread_interrupt.xml
* (add) tika-server/src/main/java/org/apache/tika/server/ServerTimeouts.java
* (add) tika-server/src/test/resources/mock/heavy_hand_100.xml
* (add) tika-server/src/test/java/org/apache/tika/server/TikaServerIntegrationTest.java
* (edit) tika-server/src/test/java/org/apache/tika/server/ServerStatusTest.java
* (edit) tika-server/src/main/java/org/apache/tika/server/resource/TikaResource.java
* (edit) tika-server/src/test/java/org/apache/tika/server/TranslateResourceTest.java
* (edit) tika-server/src/test/java/org/apache/tika/server/StackTraceTest.java
* (add) tika-server/src/test/resources/mock/system_exit.xml
* (delete) tika-server/src/main/java/org/apache/tika/server/FileCountExceededException.java
* (edit) tika-server/src/main/java/org/apache/tika/server/resource/TikaWelcome.java
* (edit) tika-server/src/test/java/org/apache/tika/server/DetectorResourceTest.java
* (add) tika-server/src/test/resources/mock/heavy_hang_30000.xml
* (edit) tika-server/src/main/java/org/apache/tika/server/resource/TranslateResource.java
* (add) tika-server/src/main/java/org/apache/tika/server/TikaServerWatchDog.java
* (edit) tika-server/src/test/java/org/apache/tika/server/CXFTestBase.java
* (edit) tika-server/src/main/java/org/apache/tika/server/resource/TikaVersion.java
* (edit) tika-server/src/test/java/org/apache/tika/server/StackTraceOffTest.java


> Make tika-server robust against ooms/infinite loops/memory leaks
> ----------------------------------------------------------------
>
>                 Key: TIKA-2725
>                 URL: https://issues.apache.org/jira/browse/TIKA-2725
>             Project: Tika
>          Issue Type: Task
>            Reporter: Tim Allison
>            Assignee: Tim Allison
>            Priority: Major
>             Fix For: 1.19, 2.0.0
>
>
> Currently, tika-server is vulnerable to ooms, inifinite loops and memory leaks.  I see
two ways of making it robust:
> 1) use the ForkParser
> 2) have tika-server spawn a child process that actually runs the server, put a watcher
thread in the child that will kill the child on oom/timeout/after x files.  The parent process
can then restart the child if it dies. 
> I somewhat prefer 2) so that we don't have to doubly pass the inputstream.  I propose
2), and I propose making it optional in Tika 1.x, but then the default in Tika 2.x.  We could
also add a status ping from parent to child in case the child gets caught up in stop the world
gc (h/t [~bleskes]).
> Other options/recommendations?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message