manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bisonti Mario <Mario.Biso...@vimar.com>
Subject R: FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed: null
Date Tue, 19 Jun 2018 10:30:14 GMT

Hallo Karl!

Now I found how to build (my first building … 😊 ) and now I am using:
/multiprocess-file-example-proprietary folder and I deployed into Tomcat .

I recreated the configuration that I used on binary version and I created the same job
It works !!!

I see on manifold cf.lg the error:
WARN 2018-06-19T12:11:52,476 (Worker thread '5') - JCIFS: Possibly transient exception detected
on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
        at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:563) ~[jcifs.jar:?]
        at jcifs.smb.SmbTransport.send(SmbTransport.java:663) ~[jcifs.jar:?]
        at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs.jar:?]
        at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs.jar:?]
        at jcifs.smb.SmbFile.send(SmbFile.java:775) ~[jcifs.jar:?]
        at jcifs.smb.SmbFile.open0(SmbFile.java:992) ~[jcifs.jar:?]
        at jcifs.smb.SmbFile.open(SmbFile.java:1009) ~[jcifs.jar:?]
        at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs.jar:?]
        at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
~[jcifs.jar:?]
        at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs.jar:?]
        at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs.jar:?]
        at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs.jar:?]
        at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs.jar:?]
        at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs.jar:?]
        at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2946) ~[jcifs.jar:?]
        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
[mcf-jcifs-connector.jar:?]
        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
[mcf-jcifs-connector.jar:?]
        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
[mcf-jcifs-connector.jar:?]
        at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-06-19T12:11:53,435 (Worker thread '3') - JCIFS: Possibly transient exception detected
on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
        at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:563) ~[jcifs.jar:?]
        at jcifs.smb.SmbTransport.send(SmbTransport.java:663) ~[jcifs.jar:?]
        at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs.jar:?]
        at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs.jar:?]
        at jcifs.smb.SmbFile.send(SmbFile.java:775) ~[jcifs.jar:?]
        at jcifs.smb.SmbFile.open0(SmbFile.java:992) ~[jcifs.jar:?]
        at jcifs.smb.SmbFile.open(SmbFile.java:1009) ~[jcifs.jar:?]
        at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs.jar:?]
        at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
~[jcifs.jar:?]
        at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs.jar:?]


but it doesn’t stuck no more , so for me it isn’t a problem if they are only warning!

Very very good!
Thanks for your help!

Mario


Da: Karl Wright <daddywri@gmail.com>
Inviato: martedì 19 giugno 2018 09:52
A: user@manifoldcf.apache.org
Oggetto: Re: FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed: null

Hi Mario,

You cannot patch the binary.  You must build from source to apply the patch.

The easiest way forward is to check out trunk directly (with svn) and build it.  The trunk
svn URL is https://svn.apache.org/repos/asf/manifoldcf/trunk<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsvn.apache.org%2Frepos%2Fasf%2Fmanifoldcf%2Ftrunk&data=01%7C01%7CMario.Bisonti%40vimar.com%7Ccea091a50fe347b3c2cc08d5d5b9888e%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=XzTfotr%2B8KS4wjHws3sKU79U9xgrWwfSAS4qK7Y%2F7NI%3D&reserved=0>
.

Karl


On Tue, Jun 19, 2018 at 3:35 AM Bisonti Mario <Mario.Bisonti@vimar.com<mailto:Mario.Bisonti@vimar.com>>
wrote:
Hallo.
Note that I specified the mime types on my solr output connection

Furthermore, I used the binary distribution, how cold I path it with tour fix?

I read on my job, that stuck up on 3 docs with:

WARN 2018-06-19T09:29:21,366 (Worker thread '14') - JCIFS: Possibly transient exception detected
on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
        at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:563) ~[jcifs-1.3.19.jar:?]
        at jcifs.smb.SmbTransport.send(SmbTransport.java:663) ~[jcifs-1.3.19.jar:?]
        at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.19.jar:?]
        at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.19.jar:?]
        at jcifs.smb.SmbFile.send(SmbFile.java:775) ~[jcifs-1.3.19.jar:?]
        at jcifs.smb.SmbFile.open0(SmbFile.java:992) ~[jcifs-1.3.19.jar:?]
        at jcifs.smb.SmbFile.open(SmbFile.java:1009) ~[jcifs-1.3.19.jar:?]
        at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.19.jar:?]
        at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
~[jcifs-1.3.19.jar:?]
        at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.19.jar:?]
        at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.19.jar:?]
        at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.19.jar:?]
        at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.19.jar:?]
        at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.19.jar:?]
        at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2946) ~[jcifs-1.3.19.jar:?]
        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
[mcf-jcifs-connector.jar:?]
        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
[mcf-jcifs-connector.jar:?]
        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
[mcf-jcifs-connector.jar:?]
        at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
FATAL 2018-06-19T09:29:21,934 (Worker thread '12') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:21,982 (Worker thread '1') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:22,546 (Worker thread '24') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,416 (Worker thread '9') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,433 (Worker thread '45') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,459 (Worker thread '17') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,489 (Worker thread '26') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,507 (Worker thread '19') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:24,516 (Worker thread '13') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:26,556 (Worker thread '2') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:26,557 (Worker thread '41') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:26,597 (Worker thread '0') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:28,591 (Worker thread '16') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:28,593 (Worker thread '33') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:28,646 (Worker thread '25') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:30,635 (Worker thread '7') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:30,635 (Worker thread '23') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:30,698 (Worker thread '8') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:32,682 (Worker thread '48') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:32,686 (Worker thread '30') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:32,736 (Worker thread '46') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:34,739 (Worker thread '29') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:34,742 (Worker thread '3') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:34,797 (Worker thread '28') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:36,789 (Worker thread '21') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:36,791 (Worker thread '11') - Error tossed: null
java.lang.NullPointerException
FATAL 2018-06-19T09:29:36,849 (Worker thread '42') - Error tossed: null


Thanks a lot for your great help

Da: Karl Wright <daddywri@gmail.com<mailto:daddywri@gmail.com>>
Inviato: lunedì 18 giugno 2018 20:39
A: user@manifoldcf.apache.org<mailto:user@manifoldcf.apache.org>
Oggetto: Re: FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed: null

Created CONNECTORS-1510 and committed a fix.
Karl

On Mon, Jun 18, 2018 at 2:33 PM Karl Wright <daddywri@gmail.com<mailto:daddywri@gmail.com>>
wrote:
It certainly is a particular file -- the mime type is null, and that's causing this line to
blow up:
    final String lowerMimeType = mimeType.toLowerCase(Locale.ROOT);


That code was added a couple of revs back to address a different problem; it's a trivial fix:

    final String lowerMimeType = (mimeType != null)?mimeType.toLowerCase(Locale.ROOT):null;

(This is HttpPoster line 811)

Karl


On Mon, Jun 18, 2018 at 2:30 PM Steph van Schalkwyk <steph@remcam.net<mailto:steph@remcam.net>>
wrote:

Looks like a particular file may be causing this. Try to find the filanem it crashes on and
copy that to asmall crawl directory. Repeat crawl.


On Mon, Jun 18, 2018 at 11:34 AM, Bisonti Mario <Mario.Bisonti@vimar.com<mailto:Mario.Bisonti@vimar.com>>
wrote:
Hallo

I configured ManifoldCF 2.10 with Tomcat 9.0.8 and Postgres 9.3

I configured multiprocess-file-example


When I create a Job to scan a big Windows share (22000 docs word, pdf, etc,) manifoldcf crash
with the message:
        at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
FATAL 2018-06-18T18:29:23,676 (Worker thread '36') - Error tossed: null
java.lang.NullPointerException
        at org.apache.manifoldcf.agents.output.solr.HttpPoster.checkMimeTypeIndexable(HttpPoster.java:811)
~[?:?]
        at org.apache.manifoldcf.agents.output.solr.SolrConnector.checkMimeTypeIndexable(SolrConnector.java:534)
~[?:?]
        at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineCheckEntryPoint.checkMimeTypeIndexable(IncrementalIngester.java:2937)
~[mcf-agents.jar:?]
        at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineCheckFanout.checkMimeTypeIndexable(IncrementalIngester.java:2864)
~[mcf-agents.jar:?]
        at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester$PipelineObject.checkMimeTypeIndexable(IncrementalIngester.java:2589)
~[mcf-agents.jar:?]
        at org.apache.manifoldcf.agents.incrementalingest.IncrementalIngester.checkMimeTypeIndexable(IncrementalIngester.java:273)
~[mcf-agents.jar:?]
        at org.apache.manifoldcf.crawler.system.WorkerThread$ProcessActivity.checkMimeTypeIndexable(WorkerThread.java:2029)
~[mcf-pull-agent.jar:?]
        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.checkIncludeFile(SharedDriveConnector.java:1439)
~[?:?]
        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector$ProcessDocumentsFilter.accept(SharedDriveConnector.java:4874)
~[?:?]
        at jcifs.smb.SmbFile.doFindFirstNext(SmbFile.java:2016) ~[?:?]
        at jcifs.smb.SmbFile.doEnum(SmbFile.java:1741) ~[?:?]
        at jcifs.smb.SmbFile.listFiles(SmbFile.java:1718) ~[?:?]
        at jcifs.smb.SmbFile.listFiles(SmbFile.java:1707) ~[?:?]
        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.fileListFiles(SharedDriveConnector.java:2318)
~[?:?]
        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:798)
~[?:?]
        at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]


If I use a smaller windows share it works.

Note that with ManifoldCF 2.9.1 HDSQLDB and QuickStart with Jetty it worked.

What could I do?

Thanks a lot
Mario

Mime
View raw message