Hallo.

Note that I specified the mime types on my solr output connection

 

Furthermore, I used the binary distribution, how cold I path it with tour fix?

 

I read on my job, that stuck up on 3 docs with:

 

WARN 2018-06-19T09:29:21,366 (Worker thread '14') - JCIFS: Possibly transient exception detected on attempt 1 while getting share security: All pipe instances are busy.

jcifs.smb.SmbException: All pipe instances are busy.

        at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:563) ~[jcifs-1.3.19.jar:?]

        at jcifs.smb.SmbTransport.send(SmbTransport.java:663) ~[jcifs-1.3.19.jar:?]

        at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.19.jar:?]

        at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.19.jar:?]

        at jcifs.smb.SmbFile.send(SmbFile.java:775) ~[jcifs-1.3.19.jar:?]

        at jcifs.smb.SmbFile.open0(SmbFile.java:992) ~[jcifs-1.3.19.jar:?]

        at jcifs.smb.SmbFile.open(SmbFile.java:1009) ~[jcifs-1.3.19.jar:?]

        at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.19.jar:?]

        at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32) ~[jcifs-1.3.19.jar:?]

        at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.19.jar:?]

        at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.19.jar:?]

        at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.19.jar:?]

        at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.19.jar:?]

        at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.19.jar:?]

        at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2946) ~[jcifs-1.3.19.jar:?]

        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438) [mcf-jcifs-connector.jar:?]

        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221) [mcf-jcifs-connector.jar:?]

        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627) [mcf-jcifs-connector.jar:?]

        at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]

FATAL 2018-06-19T09:29:21,934 (Worker thread '12') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:21,982 (Worker thread '1') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:22,546 (Worker thread '24') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:24,416 (Worker thread '9') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:24,433 (Worker thread '45') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:24,459 (Worker thread '17') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:24,489 (Worker thread '26') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:24,507 (Worker thread '19') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:24,516 (Worker thread '13') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:26,556 (Worker thread '2') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:26,557 (Worker thread '41') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:26,597 (Worker thread '0') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:28,591 (Worker thread '16') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:28,593 (Worker thread '33') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:28,646 (Worker thread '25') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:30,635 (Worker thread '7') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:30,635 (Worker thread '23') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:30,698 (Worker thread '8') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:32,682 (Worker thread '48') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:32,686 (Worker thread '30') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:32,736 (Worker thread '46') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:34,739 (Worker thread '29') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:34,742 (Worker thread '3') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:34,797 (Worker thread '28') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:36,789 (Worker thread '21') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:36,791 (Worker thread '11') - Error tossed: null

java.lang.NullPointerException

FATAL 2018-06-19T09:29:36,849 (Worker thread '42') - Error tossed: null

 

 

Thanks a lot for your great help

 

Da: Karl Wright <daddywri@gmail.com>
Inviato: lunedì 18 giugno 2018 20:39
A: user@manifoldcf.apache.org
Oggetto: Re: FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed: null

 

Created CONNECTORS-1510 and committed a fix.

Karl

 

On Mon, Jun 18, 2018 at 2:33 PM Karl Wright <daddywri@gmail.com> wrote:

It certainly is a particular file -- the mime type is null, and that's causing this line to blow up:

    final String lowerMimeType = mimeType.toLowerCase(Locale.ROOT);

 

 

That code was added a couple of revs back to address a different problem; it's a trivial fix:

 

    final String lowerMimeType = (mimeType != null)?mimeType.toLowerCase(Locale.ROOT):null;


(This is HttpPoster line 811)

 

Karl

 

 

On Mon, Jun 18, 2018 at 2:30 PM Steph van Schalkwyk <steph@remcam.net> wrote:

 

Looks like a particular file may be causing this. Try to find the filanem it crashes on and copy that to asmall crawl directory. Repeat crawl.

 

 

On Mon, Jun 18, 2018 at 11:34 AM, Bisonti Mario <Mario.Bisonti@vimar.com> wrote:

Hallo

 

I configured ManifoldCF 2.10 with Tomcat 9.0.8 and Postgres 9.3

 

I configured multiprocess-file-example

 

 

When I create a Job to scan a big Windows share (22000 docs word, pdf, etc,) manifoldcf crash with the message:
        at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]

FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed: null

java.lang.NullPointerException

        at org.apache.manifoldcf.agents.output.solr.HttpPoster.checkMimeTypeIndexable(HttpPoster.java:811) ~[?:?]

        at org.apache.manifoldcf.agents.output.solr.SolrConnector.checkMimeTypeIndexable(SolrConnector.java:534) ~[?:?]

        at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineCheckEntryPoint.checkMimeTypeIndexable(IncrementalIngester.java:2937) ~[mcf-agents.jar:?]

        at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineCheckFanout.checkMimeTypeIndexable(IncrementalIngester.java:2864) ~[mcf-agents.jar:?]

        at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObject.checkMimeTypeIndexable(IncrementalIngester.java:2589) ~[mcf-agents.jar:?]

        at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.checkMimeTypeIndexable(IncrementalIngester.java:273) ~[mcf-agents.jar:?]

        at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.checkMimeTypeIndexable(WorkerThread.java:2029) ~[mcf-pull-agent.jar:?]

        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.checkIncludeFile(SharedDriveConnector.java:1439) ~[?:?]

        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector$ProcessDocumentsFilter.accept(SharedDriveConnector.java:4874) ~[?:?]

        at jcifs.smb.SmbFile.doFindFirstNext(SmbFile.java:2016) ~[?:?]

        at jcifs.smb.SmbFile.doEnum(SmbFile.java:1741) ~[?:?]

        at jcifs.smb.SmbFile.listFiles(SmbFile.java:1718) ~[?:?]

        at jcifs.smb.SmbFile.listFiles(SmbFile.java:1707) ~[?:?]

        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.fileListFiles(SharedDriveConnector.java:2318) ~[?:?]

        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:798) ~[?:?]

        at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]

 

 

If I use a smaller windows share it works.

 

Note that with ManifoldCF 2.9.1 HDSQLDB and QuickStart with Jetty it worked.

 

What could I do?

 

Thanks a lot

Mario