manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Status and Job Management
Date Thu, 27 Sep 2018 23:40:59 GMT
Hi Damien,

I basically wanted to see if you were using Postgresql.

Postgresql is not very efficient in counting records; it's one of
Postgresql's flaws as a database.  So in ManifoldCF, we limit the number of
records counted on the status page, displaying ">xxxx" where xxxx would be
the limit.  The limit is settable in properties.xml; you can find the
property on the "how-to-build-and-deploy" page.

So, there are three possibilities.  The first possibility is that you have
a jobqueue table that is in need of analysis.  MCF does analysis
automatically as tables grow, but it's possible that you might see some
benefit in the plan by analyzing that table.  You can tell if this has
happened because the manifoldcf.log will have log messages due to
long-running queries, including plan information.

The second possibility is that your database needs to be vacuumed.
Postgresql needs periodic maintenance.  There is automatic vacuuming that
takes place, but I suggest shutting manifoldcf down and running "VACUUM
FULL" once in a while to clean things up.

The third possibility is that you have a very large number of large jobs.
If this is true, you might want to decrease the limit parameter mentioned
above to improve time spent rendering the jobstatus page.

Thanks,
Karl


On Thu, Sep 27, 2018 at 7:20 PM Damien Collis <Damien.Collis@aas.com.au>
wrote:

> Karl,
>
>
>
> I have a single Windows 2012 standalone environment hosting ManifoldCf,
> Solr and Postgress Db:
>
> 4 x Xeon E5-2660 cores
>
> 16GB Physical Memory  (2GB Tomcat/ManifoldCf, 12GB Solr)
>
>
>
>
>
> I have attached my D:\ProgramFiles\PostgreSQL\9.3\data\postgresql.conf
> file. For convenience here are the parameters that are specifically
> mentioned in
> https://manifoldcf.apache.org/release/release-2.10/en_US/how-to-build-and-deploy.html#Configuring+a+PostgreSQL+database
>
>
>
> standard_conforming_strings = on
>
> shared_buffers = 1024MB                                            # min
> 128kB
>
> #checkpoint_segments = 300                      # in logfile segments, min
> 1, 16MB each
>
> maintenance_work_mem = 2MB
>
> # NOT FOUND tcpip_socket         true
>
> checkpoint_timeout = 15min                      # range 30s-1h
>
> datestyle = 'ISO,European'
>
> autovacuum = on
>
>
>
> I hope this is the information you require.
>
>
>
> Regards
>
>
>
>
>
> *Damien Collis *
> Team Leader – Systems Integration
> Technology & Innovation Division, Link Group
>
> Level 3, 1A Homebush Bay Drive, Rhodes NSW 2138
> *T*+61 2 9375 7909
>
>
> damien.collis@linkgroup.com <Damien.Collis@linkgroup.com>
> www.linkgroup.com
>
>
>
> *From:* Karl Wright <daddywri@gmail.com>
> *Sent:* Thursday, 27 September 2018 5:10 PM
> *To:* user@manifoldcf.apache.org
> *Subject:* Re: Status and Job Management
>
>
>
> Hi Damien,
>
>
>
> Can you describe your database setup?
>
>
>
> Karl
>
>
>
>
>
> On Thu, Sep 27, 2018 at 1:50 AM Damien Collis <Damien.Collis@aas.com.au>
> wrote:
>
> All,
>
>
>
> I am currently having trouble loading the “Status and Job Management” web
> page. I have set up a new Job but am unable to start it.
>
>
>
> Sometimes the “Status and Job Management”  will finally respond (> 30
> mins), but by then I have moved on, and when I finally check back and see
> if it has rendered, my session has timed out.
>
> I know my setup is a little under spec for what I am indexing/crawling but
> I have no requirement to improve its functional performance, I am only
> having an issue from an administrative perspective.
>
>
>
> Any pointers would be greatly appreciated.
>
>
>
>
>
> Regards
>
>
>
>
>
> *Damien Collis *
> Team Leader – Systems Integration
> Technology & Innovation Division, Link Group
>
> Level 3, 1A Homebush Bay Drive, Rhodes NSW 2138
> *T*+61 2 9375 7909
>
>
> damien.collis@linkgroup.com <Damien.Collis@linkgroup.com>
> www.linkgroup.com
>
>
>
>

Mime
View raw message