manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bisonti Mario <Mario.Biso...@vimar.com>
Subject R: FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed: null
Date Tue, 19 Jun 2018 07:35:06 GMT
Hallo.
Note that I specified the mime types on my solr output connection

Furthermore, I used the binary distribution, how cold I path it with tour fix?

I read on my job, that stuck up on 3 docs with:

WARN 2018-06-19T09:29:21,366 (Worker thread '14') - JCIFS: Possibly transient exception detected
on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
        at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:563) ~[jcifs-1.3.19.jar:?]
        at jcifs.smb.SmbTransport.send(SmbTransport.java:663) ~[jcifs-1.3.19.jar:?]
        at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.19.jar:?]
        at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.19.jar:?]
        at jcifs.smb.SmbFile.send(SmbFile.java:775) ~[jcifs-1.3.19.jar:?]
        at jcifs.smb.SmbFile.open0(SmbFile.java:992) ~[jcifs-1.3.19.jar:?]
        at jcifs.smb.SmbFile.open(SmbFile.java:1009) ~[jcifs-1.3.19.jar:?]
        at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.19.jar:?]
        at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
~[jcifs-1.3.19.jar:?]
        at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.19.jar:?]
        at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.19.jar:?]
        at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.19.jar:?]
        at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.19.jar:?]
        at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.19.jar:?]
        at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2946) ~[jcifs-1.3.19.jar:?]
        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
[mcf-jcifs-connector.jar:?]
        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
[mcf-jcifs-connector.jar:?]
        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
[mcf-jcifs-connector.jar:?]
        at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
FATAL 2018-06-19T09:29:21,934 (Worker thread '12') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:21,982 (Worker thread '1') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:22,546 (Worker thread '24') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,416 (Worker thread '9') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,433 (Worker thread '45') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,459 (Worker thread '17') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,489 (Worker thread '26') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,507 (Worker thread '19') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,516 (Worker thread '13') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:26,556 (Worker thread '2') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:26,557 (Worker thread '41') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:26,597 (Worker thread '0') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:28,591 (Worker thread '16') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:28,593 (Worker thread '33') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:28,646 (Worker thread '25') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:30,635 (Worker thread '7') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:30,635 (Worker thread '23') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:30,698 (Worker thread '8') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:32,682 (Worker thread '48') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:32,686 (Worker thread '30') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:32,736 (Worker thread '46') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:34,739 (Worker thread '29') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:34,742 (Worker thread '3') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:34,797 (Worker thread '28') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:36,789 (Worker thread '21') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:36,791 (Worker thread '11') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:36,849 (Worker thread '42') - Error tossed: null


Thanks a lot for your great help

Da: Karl Wright <daddywri@gmail.com>
Inviato: lunedì 18 giugno 2018 20:39
A: user@manifoldcf.apache.org
Oggetto: Re: FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed: null

Created CONNECTORS-1510 and committed a fix.
Karl

On Mon, Jun 18, 2018 at 2:33 PM Karl Wright <daddywri@gmail.com<mailto:daddywri@gmail.com>>
wrote:
It certainly is a particular file -- the mime type is null, and that's causing this line to
blow up:
    final String lowerMimeType = mimeType.toLowerCase(Locale.ROOT);


That code was added a couple of revs back to address a different problem; it's a trivial fix:

    final String lowerMimeType = (mimeType != null)?mimeType.toLowerCase(Locale.ROOT):null;

(This is HttpPoster line 811)

Karl


On Mon, Jun 18, 2018 at 2:30 PM Steph van Schalkwyk <steph@remcam.net<mailto:steph@remcam.net>>
wrote:

Looks like a particular file may be causing this. Try to find the filanem it crashes on and
copy that to asmall crawl directory. Repeat crawl.


On Mon, Jun 18, 2018 at 11:34 AM, Bisonti Mario <Mario.Bisonti@vimar.com<mailto:Mario.Bisonti@vimar.com>>
wrote:
Hallo

I configured ManifoldCF 2.10 with Tomcat 9.0.8 and Postgres 9.3

I configured multiprocess-file-example


When I create a Job to scan a big Windows share (22000 docs word, pdf, etc,) manifoldcf crash
with the message:
        at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed: null
java.lang.NullPointerException
        at org.apache.manifoldcf.agents.output.solr.HttpPoster.checkMimeTypeIndexable(HttpPoster.java:811)
~[?:?]
        at org.apache.manifoldcf.agents.output.solr.SolrConnector.checkMimeTypeIndexable(SolrConnector.java:534)
~[?:?]
        at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineCheckEntryPoint.checkMimeTypeIndexable(IncrementalIngester.java:2937)
~[mcf-agents.jar:?]
        at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineCheckFanout.checkMimeTypeIndexable(IncrementalIngester.java:2864)
~[mcf-agents.jar:?]
        at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObject.checkMimeTypeIndexable(IncrementalIngester.java:2589)
~[mcf-agents.jar:?]
        at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.checkMimeTypeIndexable(IncrementalIngester.java:273)
~[mcf-agents.jar:?]
        at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.checkMimeTypeIndexable(WorkerThread.java:2029)
~[mcf-pull-agent.jar:?]
        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.checkIncludeFile(SharedDriveConnector.java:1439)
~[?:?]
        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector$ProcessDocumentsFilter.accept(SharedDriveConnector.java:4874)
~[?:?]
        at jcifs.smb.SmbFile.doFindFirstNext(SmbFile.java:2016) ~[?:?]
        at jcifs.smb.SmbFile.doEnum(SmbFile.java:1741) ~[?:?]
        at jcifs.smb.SmbFile.listFiles(SmbFile.java:1718) ~[?:?]
        at jcifs.smb.SmbFile.listFiles(SmbFile.java:1707) ~[?:?]
        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.fileListFiles(SharedDriveConnector.java:2318)
~[?:?]
        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:798)
~[?:?]
        at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]


If I use a smaller windows share it works.

Note that with ManifoldCF 2.9.1 HDSQLDB and QuickStart with Jetty it worked.

What could I do?

Thanks a lot
Mario

Mime
View raw message