manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Advice on which PostgreSQL to use with ManifoldCF 2.6
Date Fri, 03 Mar 2017 17:35:28 GMT
Hi Guy:

It is expected that sometimes database deadlock will develop, and the
transaction will need to be retried.  There is code in MCF for doing this:

>>>>>>
    if (sqlState != null && sqlState.equals("40001"))
      return new
ManifoldCFException(message,e,ManifoldCFException.DATABASE_TRANSACTION_ABORT);
<<<<<<

I suspect that your version of MCF is old enough so that this particular
error and the associated retry are not taking place.  Upgrading to 2.6 will
definitely help there.

Karl


On Fri, Mar 3, 2017 at 11:17 AM, Standen Guy <Guy.Standen@uk.fujitsu.com>
wrote:

> Hi Karl,
>
>                 Thanks for coming back so quickly. Unfortunately I wasn’t
> using a JCIFS connection. One of the issues I was seeing was between a
> crawl of an intranet site (no explicit throttling other than number of
> connections) and   scheduled crawl (every 5 mins) to a relational DB via
> JDBC connector again no explicit throttling.  To simplify things both jobs
> are using a NULL output connection. Sometimes both the Web crawl and the
> JDBC connection can run together but at other times 1 or both jobs will
> appear to lock up with just a few active documents showing. When I get a
> lock up the mcf log contains errors like:
>
>
>
> “DEBUG 2017-03-03 15:28:20,466 (Worker thread '5') - Exception Database
> exception: SQLException doing query (40001): ERROR: could not serialize
> access due to read/write dependencies among transactions”
>
>
>
> See the attached log extract for a little more detail.
>
>
>
> Any view why this might be happening?
>
>
>
> Best Regards,
>
>
>
> Guy
>
>
>
>
>
>
>
>
>
>
>
>
>
> *From:* Karl Wright [mailto:daddywri@gmail.com]
> *Sent:* 03 March 2017 11:27
> *To:* user@manifoldcf.apache.org
> *Subject:* Re: Advice on which PostgreSQL to use with ManifoldCF 2.6
>
>
>
> Hi Guy,
>
>
>
> A issue with concurrent jobs is known for jobs sharing the same JCIFS
> connection.  Is that what you are using?  This has nothing to do with the
> version of Postgresql you are using; it has to do with what "bins"
> documents are thought to come from.  There has been a recent improvement
> for this issue, which will be released in April.  See
> https://issues.apache.org/jira/browse/CONNECTORS-1364.
>
>
>
> The current version of MCF (2.6) supports Solr 6.x.
>
>
>
> Thanks,
>
> Karl
>
>
>
>
>
> On Fri, Mar 3, 2017 at 5:27 AM, Standen Guy <Guy.Standen@uk.fujitsu.com>
> wrote:
>
> Hi Karl,
>
> I am currently using MCF 2.0.1 with PostgreSQL 9.3.5 on Windows and have
> had some issues with multiple jobs running concurrently.
>
> I am considering upgrading to MCF 2.6 and to a newer version of
> PostgreSQL. Would you be able to advise which version of PostgreSQL I
> should consider using with MCF 2.6 (e.g.  PostgreSQL  9.3.16 or 9.6.2)
>
>
>
> I am also considering upgrading from SOLR 4.10.3 to a newer version. The
> MCF compatibility matrix mentions that compatibility has been tested to
> SOLR version 4.5.1. Do you have any advice about compatibility with the
> newer versions of SOLR e.g. 6.4.1.
>
>
>
> Best Regards
>
>
>
> Guy
>
>
>
>
> Unless otherwise stated, this email has been sent from Fujitsu Services
> Limited (registered in England No 96056); Fujitsu EMEA PLC (registered in
> England No 2216100) both with registered offices at: 22 Baker Street,
> London W1U 3BW; PFU (EMEA) Limited, (registered in England No 1578652) and
> Fujitsu Laboratories of Europe Limited (registered in England No. 4153469)
> both with registered offices at: Hayes Park Central, Hayes End Road, Hayes,
> Middlesex, UB4 8FE.
> This email is only for the use of its intended recipient. Its contents are
> subject to a duty of confidence and may be privileged. Fujitsu does not
> guarantee that this email has not been intercepted and amended or that it
> is virus-free.
>
>
>
> Unless otherwise stated, this email has been sent from Fujitsu Services
> Limited (registered in England No 96056); Fujitsu EMEA PLC (registered in
> England No 2216100) both with registered offices at: 22 Baker Street,
> London W1U 3BW; PFU (EMEA) Limited, (registered in England No 1578652) and
> Fujitsu Laboratories of Europe Limited (registered in England No. 4153469)
> both with registered offices at: Hayes Park Central, Hayes End Road, Hayes,
> Middlesex, UB4 8FE.
> This email is only for the use of its intended recipient. Its contents are
> subject to a duty of confidence and may be privileged. Fujitsu does not
> guarantee that this email has not been intercepted and amended or that it
> is virus-free.
>

Mime
View raw message