manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Priya Arora <pr...@smartshore.nl>
Subject Fwd: Manifold Crawler Crashes
Date Thu, 20 Jun 2019 06:04:00 GMT
Hi,

I am running multiple jobs(2,3) simultaneously on Manifold server and the
configuration is

1) For Crawler server - 16 GB RAM and 8-Core Intel(R) Xeon(R) CPU E5-2660
v3 @ 2.60GHz and

2) For Elasticsearch server - 48GB and 1-Core Intel(R) Xeon(R) CPU E5-2660
v3 @ 2.60GHz
Job working is to fetch data from some public and intranet sites and then
ingesting data into Elastic search.

Maximum connection on both Repository connections and Output connection is
48(for all 3 jobs).

What problem i am facing here is when i am running multiple jobs the
manifold crashes after some time and there is nothing inside manifold.log
files that hints out me some error.
Is the maximum connections increases(48+48+48) while running all three jobs
together?
So do i need to divide max connections(48) among all three jobs?
How many connections maximum we can have to run the jobs individually and
simultaneously.

what should be the maximum allowed number of max handles in properties.xml
file and postgres config file?

So the problem is to figure out what is the reason for the crawler crash.
Can you please help me on that as soon as possible.

Thanks and regards
Priya
priya@smartshore.nl

Mime
View raw message