manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Markus Schuch <markus_sch...@web.de>
Subject Re: Manifoldcf server Error
Date Fri, 20 Dec 2019 08:53:24 GMT
Hi Priya,

your manifoldcf JVM suffers from high garbage collection pressure:

    java.lang.OutOfMemoryError: GC overhead limit exceeded

What is your current heap size?
Without knowing that, i suggest to increase the heap size. (java -Xmx...)

Cheers,
Markus

Am 20.12.2019 um 09:02 schrieb Priya Arora:
> Hi All,
>
> I am facing below error while accessing Manifoldcf. Requirement is to
> crawl data from a website using Repository as "Web" and Output connector
> as "Elastic Search"
> Manifoldcf is configured inside a docker container and also postgres is
> used a docker container.
> When launching manifold getting below error
> image.png
>
> When checked logs:-
> *1)sudo docker exec -it 0b872dfafc5c tail -1000
> /usr/share/manifoldcf/example/logs/manifoldcf.log*
> FATAL 2019-12-20T06:06:13,176 (Stuffer thread) - Error tossed: Timer
> already cancelled.
> java.lang.IllegalStateException: Timer already cancelled.
>         at java.util.Timer.sched(Timer.java:397) ~[?:1.8.0_232]
>         at java.util.Timer.schedule(Timer.java:193) ~[?:1.8.0_232]
>         at
> org.postgresql.jdbc.PgConnection.addTimerTask(PgConnection.java:1113)
> ~[postgresql-42.1.3.jar:42.1.3]
>         at
> org.postgresql.jdbc.PgStatement.startTimer(PgStatement.java:887)
> ~[postgresql-42.1.3.jar:42.1.3]
>         at
> org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:427)
> ~[postgresql-42.1.3.jar:42.1.3]
>         at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:354)
> ~[postgresql-42.1.3.jar:42.1.3]
>         at
> org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:169)
> ~[postgresql-42.1.3.jar:42.1.3]
>         at
> org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:136)
> ~[postgresql-42.1.3.jar:42.1.3]
>         at
> org.postgresql.jdbc.PgConnection.isValid(PgConnection.java:1311)
> ~[postgresql-42.1.3.jar:42.1.3]
>         at
> org.apache.manifoldcf.core.jdbcpool.ConnectionPool.getConnection(ConnectionPool.java:92)
> ~[mcf-core.jar:?]
>         at
> org.apache.manifoldcf.core.database.ConnectionFactory.getConnectionWithRetries(ConnectionFactory.java:126)
> ~[mcf-core.jar:?]
>         at
> org.apache.manifoldcf.core.database.ConnectionFactory.getConnection(ConnectionFactory.java:75)
> ~[mcf-core.jar:?]
>         at
> org.apache.manifoldcf.core.database.Database.executeUncachedQuery(Database.java:797)
> ~[mcf-core.jar:?]
>         at
> org.apache.manifoldcf.core.database.Database$QueryCacheExecutor.create(Database.java:1457)
> ~[mcf-core.jar:?]
>         at
> org.apache.manifoldcf.core.cachemanager.CacheManager.findObjectsAndExecute(CacheManager.java:146)
> ~[mcf-core.jar:?]
>         at
> org.apache.manifoldcf.core.database.Database.executeQuery(Database.java:204)
> ~[mcf-core.jar:?]
>         at
> org.apache.manifoldcf.core.database.DBInterfacePostgreSQL.performQuery(DBInterfacePostgreSQL.java:837)
> ~[mcf-core.jar:?]
>         at
> org.apache.manifoldcf.core.database.BaseTable.performQuery(BaseTable.java:221)
> ~[mcf-core.jar:?]
>         at
> org.apache.manifoldcf.crawler.jobs.Jobs.getActiveJobConnections(Jobs.java:736)
> ~[mcf-pull-agent.jar:?]
>         at
> org.apache.manifoldcf.crawler.jobs.JobManager.getNextDocuments(JobManager.java:2869)
> ~[mcf-pull-agent.jar:?]
>         at
> org.apache.manifoldcf.crawler.system.StufferThread.run(StufferThread.java:186)
> [mcf-pull-agent.jar:?]
> *2)sudo docker logs <CID> --tail 1000*
> Exception in thread "PostgreSQL-JDBC-SharedTimer-1"
> java.lang.OutOfMemoryError: GC overhead limit exceeded
>         at java.util.ArrayList.iterator(ArrayList.java:840)
>         at
> java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1316)
>         at java.net.InetAddress.getAllByName0(InetAddress.java:1277)
>         at java.net.InetAddress.getAllByName(InetAddress.java:1193)
>         at java.net.InetAddress.getAllByName(InetAddress.java:1127)
>         at java.net.InetAddress.getByName(InetAddress.java:1077)
>         at java.net.InetSocketAddress.<init>(InetSocketAddress.java:220)
>         at org.postgresql.core.PGStream.<init>(PGStream.java:66)
>         at
> org.postgresql.core.QueryExecutorBase.sendQueryCancel(QueryExecutorBase.java:155)
>         at
> org.postgresql.jdbc.PgConnection.cancelQuery(PgConnection.java:971)
>         at org.postgresql.jdbc.PgStatement.cancel(PgStatement.java:812)
>         at org.postgresql.jdbc.PgStatement$1.run(PgStatement.java:880)
>         at java.util.TimerThread.mainLoop(Timer.java:555)
>         at java.util.TimerThread.run(Timer.java:505)
> 2019-12-19 18:09:05,848 Job start thread ERROR Unable to write to stream
> logs/manifoldcf.log for appender MyFile
> 2019-12-19 18:09:05,848 Seeding thread ERROR Unable to write to stream
> logs/manifoldcf.log for appender MyFile
> 2019-12-19 18:09:05,848 Job reset thread ERROR Unable to write to stream
> logs/manifoldcf.log for appender MyFile
> 2019-12-19 18:09:05,848 Job notification thread ERROR Unable to write to
> stream logs/manifoldcf.log for appender MyFile
> 2019-12-19 18:09:05,849 Seeding thread ERROR An exception occurred
> processing Appender MyFile
> org.apache.logging.log4j.core.appender.AppenderLoggingException: Error
> flushing stream logs/manifoldcf.log
>         at
> org.apache.logging.log4j.core.appender.OutputStreamManager.flush(OutputStreamManager.java:159).
>
> _Also tried the approach to clean up Database by truncating all
> manifoldcf related tables, but still getting this error._
>
> Parameters defined in *postgresql conf *file is as suggested :- and
> "max_pred_per_locks_transctions" is set to value "256".
> image.png

Mime
View raw message