manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Job stuck without message
Date Wed, 28 Nov 2018 15:35:55 GMT
Another thing you could do is get a thread dump of the agents process.

Karl


On Wed, Nov 28, 2018 at 10:35 AM Karl Wright <daddywri@gmail.com> wrote:

> Can you look into the database jobqueue table and provide a row that
> corresponds to one of these documents?
>
> Thanks,
> Karl
>
>
> On Wed, Nov 28, 2018 at 10:26 AM Bisonti Mario <Mario.Bisonti@vimar.com>
> wrote:
>
>> Hallo.
>>
>> Repository has Max connection=10
>>
>>
>>
>> In the Document Status report” I see many item with :
>>
>> State=“Not yet processed”
>>
>> Status=”Ready for processing”
>>
>> Scheduled=01-01-1970 01:00:00.000”
>>
>> Scheduled Action=”Process”
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> But the job no more walk..
>>
>>
>>
>>
>>
>> *Da:* Karl Wright <daddywri@gmail.com>
>> *Inviato:* mercoledì 28 novembre 2018 16:03
>> *A:* user@manifoldcf.apache.org
>> *Oggetto:* Re: Job stuck without message
>>
>>
>>
>> "Pipe instances are busy" occurs because you are overloading the SMB
>> access to your servers.  How many connections do you have allocated for
>> your repository connection?  You probably want to limit this to 2-3 if you
>> see this error a lot, and it appears you do.
>>
>> " Tika Server: Tika Server rejects: Tika Server rejected document with
>> the following reason: Unprocessable Entity" means the document is not
>> properly formed XML.  The rejection will mean the document isn't indexed,
>> but this will not stop the job.
>>
>> If nothing is happening and you don't know why, I'd suggest looking at
>> the Document Status report to figure out what documents are not being
>> processed and why.  It is quite possible they are all in the process of
>> being retried because of the "Pipe instances" issue above.
>>
>>
>>
>> Karl
>>
>>
>>
>> On Wed, Nov 28, 2018 at 9:46 AM Bisonti Mario <Mario.Bisonti@vimar.com>
>> wrote:
>>
>> Hallo Karl.
>>
>> I take this ticket because,now, after I use zookeeper, my job works for 7
>> hours and now it is in hang status.
>>
>> I see running but it seems hanging, no log from 1 hour
>>
>>
>>
>> This is the last manifoldcf.log lines:
>>
>>
>>
>>
>>
>>         at jcifs.smb.SmbFile.open(SmbFile.java:1010)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446)
>> [mcf-jcifs-connector.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222)
>> [mcf-jcifs-connector.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
>> [mcf-jcifs-connector.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
>> [mcf-pull-agent.jar:?]
>>
>> WARN 2018-11-28T14:46:21,524 (Worker thread '59') - JCIFS: Possibly
>> transient exception detected on attempt 1 while getting share security: All
>> pipe instances are busy.
>>
>> jcifs.smb.SmbException: All pipe instances are busy.
>>
>>         at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbSession.send(SmbSession.java:238)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbTree.send(SmbTree.java:119)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.send(SmbFile.java:776)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.open0(SmbFile.java:993)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.open(SmbFile.java:1010)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446)
>> [mcf-jcifs-connector.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222)
>> [mcf-jcifs-connector.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
>> [mcf-jcifs-connector.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
>> [mcf-pull-agent.jar:?]
>>
>> WARN 2018-11-28T14:46:25,232 (Worker thread '45') - Tika Server: Tika
>> Server rejects: Tika Server rejected document with the following reason:
>> Unprocessable Entity
>>
>> WARN 2018-11-28T14:46:25,277 (Worker thread '70') - Tika Server: Tika
>> Server rejects: Tika Server rejected document with the following reason:
>> Unprocessable Entity
>>
>> WARN 2018-11-28T14:46:28,825 (Worker thread '27') - JCIFS: Possibly
>> transient exception detected on attempt 1 while getting share security: All
>> pipe instances are busy.
>>
>> jcifs.smb.SmbException: All pipe instances are busy.
>>
>>         at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbSession.send(SmbSession.java:238)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbTree.send(SmbTree.java:119)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.send(SmbFile.java:776)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.open0(SmbFile.java:993)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.open(SmbFile.java:1010)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446)
>> [mcf-jcifs-connector.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222)
>> [mcf-jcifs-connector.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
>> [mcf-jcifs-connector.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
>> [mcf-pull-agent.jar:?]
>>
>> WARN 2018-11-28T14:46:40,473 (Worker thread '79') - JCIFS: Possibly
>> transient exception detected on attempt 1 while getting share security: All
>> pipe instances are busy.
>>
>> jcifs.smb.SmbException: All pipe instances are busy.
>>
>>         at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbSession.send(SmbSession.java:238)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbTree.send(SmbTree.java:119)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.send(SmbFile.java:776)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.open0(SmbFile.java:993)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.open(SmbFile.java:1010)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446)
>> [mcf-jcifs-connector.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222)
>> [mcf-jcifs-connector.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
>> [mcf-jcifs-connector.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
>> [mcf-pull-agent.jar:?]
>>
>> WARN 2018-11-28T14:46:54,855 (Worker thread '65') - JCIFS: Possibly
>> transient exception detected on attempt 1 while getting share security: All
>> pipe instances are busy.
>>
>> jcifs.smb.SmbException: All pipe instances are busy.
>>
>>         at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbSession.send(SmbSession.java:238)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbTree.send(SmbTree.java:119)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.send(SmbFile.java:776)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.open0(SmbFile.java:993)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.open(SmbFile.java:1010)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446)
>> [mcf-jcifs-connector.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222)
>> [mcf-jcifs-connector.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
>> [mcf-jcifs-connector.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
>> [mcf-pull-agent.jar:?]
>>
>> WARN 2018-11-28T14:46:55,175 (Worker thread '83') - JCIFS: Possibly
>> transient exception detected on attempt 1 while getting share security: All
>> pipe instances are busy.
>>
>> jcifs.smb.SmbException: All pipe instances are busy.
>>
>>         at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbSession.send(SmbSession.java:238)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbTree.send(SmbTree.java:119)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.send(SmbFile.java:776)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.open0(SmbFile.java:993)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.open(SmbFile.java:1010)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446)
>> [mcf-jcifs-connector.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222)
>> [mcf-jcifs-connector.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
>> [mcf-jcifs-connector.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
>> [mcf-pull-agent.jar:?]
>>
>>
>>
>>
>>
>>
>>
>> I don’t know what to check.
>>
>>
>>
>> Tika server is ok, and it doesn’t more restart it .
>>
>>
>>
>>
>>
>>
>>
>> *Da:* Karl Wright <daddywri@gmail.com>
>> *Inviato:* martedì 6 novembre 2018 15:27
>> *A:* user@manifoldcf.apache.org
>> *Oggetto:* Re: Job stuck without message
>>
>>
>>
>> I added a couple of questions to the ticket.  Please reply.
>>
>>
>>
>> Thanks,
>>
>> Karl
>>
>>
>>
>>
>>
>> On Tue, Nov 6, 2018 at 8:56 AM Bisonti Mario <Mario.Bisonti@vimar.com>
>> wrote:
>>
>> Thanks a lot, Karl.
>>
>> I created a ticket.
>>
>> https://issues.apache.org/jira/browse/CONNECTORS-1554
>> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FCONNECTORS-1554&data=01%7C01%7CMario.Bisonti%40vimar.com%7C76564dfb521a4a175ef608d655429b6b%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=H2Ps6N6MQHUo72mKEs%2BX0xSHtonLkXTNG%2FEA3c5qbUc%3D&reserved=0>
>>
>>
>>
>>
>>
>> Thanks
>>
>>
>>
>> Mario
>>
>>
>>
>>
>>
>>
>>
>> *Da:* Karl Wright <daddywri@gmail.com>
>> *Inviato:* martedì 6 novembre 2018 14:28
>> *A:* user@manifoldcf.apache.org
>> *Oggetto:* Re: Job stuck without message
>>
>>
>>
>> ok, can you create a ticket?  Also, I'd appreciate it if you can look at
>> the simple history for one of these documents; I need to see what happened
>> to it last.
>>
>>
>>
>> Thanks,
>>
>> Karl
>>
>>
>>
>>
>>
>> On Tue, Nov 6, 2018 at 7:32 AM Bisonti Mario <Mario.Bisonti@vimar.com>
>> wrote:
>>
>> My version is 2.11
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *Da:* Karl Wright <daddywri@gmail.com>
>> *Inviato:* martedì 6 novembre 2018 13:07
>> *A:* user@manifoldcf.apache.org
>> *Oggetto:* Re: Job stuck without message
>>
>>
>>
>> Thanks.
>>
>> What version of ManifoldCF are you using?  We fixed a problem a while
>> back having to do with documents that (because of error processing) get put
>> into a "ready for processing" state which don't have any document priority
>> set.  But this should have been addressed, certainly, by the most recent
>> release and probably by 2.10 as well.
>>
>>
>>
>> Karl
>>
>>
>>
>>
>>
>> On Tue, Nov 6, 2018 at 5:43 AM Bisonti Mario <Mario.Bisonti@vimar.com>
>> wrote:
>>
>> Hallo Karl.
>>
>> When it hangs I see in the Queue status:
>>
>>
>>
>> And in the Document Status:
>>
>>
>>
>>
>>
>> *Da:* Karl Wright <daddywri@gmail.com>
>> *Inviato:* martedì 30 ottobre 2018 19:32
>> *A:* user@manifoldcf.apache.org
>> *Oggetto:* Re: Job stuck without message
>>
>>
>>
>> What I am interested in now is the Document Status report for any one of
>> the documents that is 'stuck'.  The next crawl time value is the critical
>> field.  Can you include an example?
>>
>>
>>
>> Karl
>>
>>
>>
>> On Tue, Oct 30, 2018, 12:36 PM Bisonti Mario <Mario.Bisonti@vimar.com>
>> wrote:
>>
>> Thanks a lot, Karl.
>>
>>
>>
>> It happens that the job starts, it works and index for an hour and after
>> it frezzes, I haven’t error or waiting status in Document Queue o Simple
>> History, I have only “OK” status so, I haven’t failures.
>>
>>
>>
>> I am not able to see other log errors other from the manifoldcf.log
>>
>>
>>
>> Solr server is ok
>>
>> Tika server is ok
>>
>> Agent is ok
>>
>> Tomcat with ManifoldCF is ok
>>
>>
>>
>> I could search if I could to put in info log mode for example Tika
>> servrer or Solr.
>>
>>
>>
>> Thanks..
>>
>>
>>
>>
>>
>> *Da:* Karl Wright <daddywri@gmail.com>
>> *Inviato:* martedì 30 ottobre 2018 16:38
>> *A:* user@manifoldcf.apache.org
>> *Oggetto:* Re: Job stuck without message
>>
>>
>>
>> Hi Mario,
>>
>> Please look at the Queue Status report to determine what is waiting and
>> why it is waiting.
>> You can also look at the Simple History to see what has been happening.
>> If you are getting 100% failures in fetching documents then you may need to
>> address this because your infrastructure is unhappy.  If the failure is
>> something that indicates that the document is never going to be readable,
>> that's a different problem and we might need to address that in the
>> connector.
>>
>>
>>
>> Karl
>>
>>
>>
>>
>>
>> On Tue, Oct 30, 2018 at 10:33 AM Bisonti Mario <Mario.Bisonti@vimar.com>
>> wrote:
>>
>>
>>
>> Thanks a lot Karl
>>
>>
>>
>> Yes, I see many docs in the docs queue but they are inactive.
>>
>>
>>
>> Infact i see that no more docs are indexed in Solr and I see that job is
>> with the same number of docs Active (35012)
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *Da:* Karl Wright <daddywri@gmail.com>
>> *Inviato:* martedì 30 ottobre 2018 13:59
>> *A:* user@manifoldcf.apache.org
>> *Oggetto:* Re: Job stuck without message
>>
>>
>>
>> The reason the job is "stuck" is because:
>>
>> ' JCIFS: Possibly transient exception detected on attempt 1 while
>> getting share security: All pipe instances are busy.'
>>
>> This means that ManifoldCF will retry this document for a while before it
>> gives up on it.  It appears to be stuck but it is not.  You can verify that
>> by looking at the Document Queue report to see what is queued and what
>> times the various documents will be retried.
>>
>>
>>
>> Karl
>>
>>
>>
>>
>>
>> On Tue, Oct 30, 2018 at 5:07 AM Bisonti Mario <Mario.Bisonti@vimar.com>
>> wrote:
>>
>> Hallo.
>>
>>
>>
>> I started a job that works for some minutes, and after it stucks.
>>
>>
>>
>> In the manifoldcf.log I see:
>>         at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
>> [mcf-jcifs-connector.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
>> [mcf-pull-agent.jar:?]
>>
>> WARN 2018-10-30T09:21:31,440 (Worker thread '2') - Tika Server: Tika
>> Server rejects: Tika Server rejected document with the following reason:
>> Unprocessable Entity
>>
>> WARN 2018-10-30T09:21:33,502 (Worker thread '14') - Tika Server: Tika
>> Server rejects: Tika Server rejected document with the following reason:
>> Unprocessable Entity
>>
>> WARN 2018-10-30T09:21:37,725 (Worker thread '30') - Tika Server: Tika
>> Server rejects: Tika Server rejected document with the following reason:
>> Unprocessable Entity
>>
>> WARN 2018-10-30T09:21:44,406 (Worker thread '49') - Tika Server: Tika
>> Server rejects: Tika Server rejected document with the following reason:
>> Unprocessable Entity
>>
>> WARN 2018-10-30T09:21:47,310 (Worker thread '15') - Tika Server: Tika
>> Server rejects: Tika Server rejected document with the following reason:
>> Unprocessable Entity
>>
>> WARN 2018-10-30T09:21:52,000 (Worker thread '27') - Tika Server: Tika
>> Server rejects: Tika Server rejected document with the following reason:
>> Unprocessable Entity
>>
>> WARN 2018-10-30T09:21:53,526 (Worker thread '15') - Tika Server: Tika
>> Server rejects: Tika Server rejected document with the following reason:
>> Unprocessable Entity
>>
>> WARN 2018-10-30T09:22:04,511 (Worker thread '3') - JCIFS: Possibly
>> transient exception detected on attempt 1 while getting share security: All
>> pipe instances are busy.
>>
>> jcifs.smb.SmbException: All pipe instances are busy.
>>
>>         at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbSession.send(SmbSession.java:238)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbTree.send(SmbTree.java:119)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.send(SmbFile.java:776)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.open0(SmbFile.java:993)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.open(SmbFile.java:1010)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
>> ~[jcifs-1.3.18.3.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
>> [mcf-jcifs-connector.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
>> [mcf-jcifs-connector.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
>> [mcf-jcifs-connector.jar:?]
>>
>>         at
>> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
>> [mcf-pull-agent.jar:?]
>>
>> WARN 2018-10-30T09:22:10,359 (Worker thread '27') - Tika Server: Tika
>> Server rejects: Tika Server rejected document with the following reason:
>> Unprocessable Entity
>>
>> WARN 2018-10-30T09:22:13,932 (Worker thread '12') - Tika Server: Tika
>> Server rejects: Tika Server rejected document with the following reason:
>> Unprocessable Entity
>>
>> WARN 2018-10-30T09:22:14,274 (Worker thread '23') - Tika Server: Tika
>> Server rejects: Tika Server rejected document with the following reason:
>> Unprocessable Entity
>>
>> WARN 2018-10-30T09:22:19,933 (Worker thread '8') - Tika Server: Tika
>> Server rejects: Tika Server rejected document with the following reason:
>> Unprocessable Entity
>>
>> WARN 2018-10-30T09:23:59,920 (Worker thread '39') - Tika Server: Tika
>> Server rejects: Tika Server rejected document with the following reason:
>> Unprocessable Entity
>>
>> WARN 2018-10-30T09:24:09,059 (Worker thread '43') - Tika Server: Tika
>> Server rejects: Tika Server rejected document with the following reason:
>> Unprocessable Entity
>>
>>
>>
>>
>>
>>
>>
>> What could I check?
>>
>>
>>
>> Tika server works as standalone.
>>
>>
>>
>> Could you help  me?
>>
>>
>>
>> Thanks a lot
>>
>>
>>
>> Mario
>>
>>
>>
>>

Mime
View raw message