manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Job stuck without message
Date Wed, 28 Nov 2018 20:51:58 GMT
The database row indicates there is no reason that the document should not
be queued and processed.
As for getting a thread dump, there's a "force" option (-F).

The only other reason stuff may not run is if the query plan for
identifying documents to process has gone horribly wrong.  We should see
that in the thread dump however.

I will unfortunately need to be offline for the next 24 hours due to an
emergency situation, but if it turns out that your agents process is busy
executing a long-running query, then I suggest analyzing the jobqueue table
to get a better plan.  This happens automatically but there are conditions
under which it doesn't happen frequently enough.  If the job is waiting for
locks, then the stack trace will tell me where.

Thanks,
Karl
Karl


On Wed, Nov 28, 2018 at 11:07 AM Bisonti Mario <Mario.Bisonti@vimar.com>
wrote:

> I attatched  a row that correspond to a row of one of these documents in
> this mail
>
>
>
>
>
>
>
> I obtain the pid of:
> "/bin/bash -e
> /opt/manifoldcf/multiprocess-zk-example-proprietary/start-agents.sh"
>
> The pid is 1233
>
>
>
> I tried to use
>
> sudo jstack -l 1233 > /tmp/jstack_start_agent.log
>
>
>
> but I obtain:
>
> 1233: Unable to open socket file /proc/1233/cwd/.attach_pid1233: target
> process 1233 doesn't respond within 10500ms or HotSpot VM not loaded
>
>
>
> Perhaps isn’t it the right way to obtain a thread dump?
>
> Excuse me but I am not a Linux expert..
>
>
>
>
>
>
>
>
>
> *Da:* Karl Wright <daddywri@gmail.com>
> *Inviato:* mercoledì 28 novembre 2018 16:36
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Job stuck without message
>
>
>
> Another thing you could do is get a thread dump of the agents process.
>
>
>
> Karl
>
>
>
>
>
> On Wed, Nov 28, 2018 at 10:35 AM Karl Wright <daddywri@gmail.com> wrote:
>
> Can you look into the database jobqueue table and provide a row that
> corresponds to one of these documents?
>
>
>
> Thanks,
>
> Karl
>
>
>
>
>
> On Wed, Nov 28, 2018 at 10:26 AM Bisonti Mario <Mario.Bisonti@vimar.com>
> wrote:
>
> Hallo.
>
> Repository has Max connection=10
>
>
>
> In the Document Status report” I see many item with :
>
> State=“Not yet processed”
>
> Status=”Ready for processing”
>
> Scheduled=01-01-1970 01:00:00.000”
>
> Scheduled Action=”Process”
>
>
>
>
>
>
>
>
>
> But the job no more walk..
>
>
>
>
>
> *Da:* Karl Wright <daddywri@gmail.com>
> *Inviato:* mercoledì 28 novembre 2018 16:03
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Job stuck without message
>
>
>
> "Pipe instances are busy" occurs because you are overloading the SMB
> access to your servers.  How many connections do you have allocated for
> your repository connection?  You probably want to limit this to 2-3 if you
> see this error a lot, and it appears you do.
>
> " Tika Server: Tika Server rejects: Tika Server rejected document with
> the following reason: Unprocessable Entity" means the document is not
> properly formed XML.  The rejection will mean the document isn't indexed,
> but this will not stop the job.
>
> If nothing is happening and you don't know why, I'd suggest looking at the
> Document Status report to figure out what documents are not being processed
> and why.  It is quite possible they are all in the process of being retried
> because of the "Pipe instances" issue above.
>
>
>
> Karl
>
>
>
> On Wed, Nov 28, 2018 at 9:46 AM Bisonti Mario <Mario.Bisonti@vimar.com>
> wrote:
>
> Hallo Karl.
>
> I take this ticket because,now, after I use zookeeper, my job works for 7
> hours and now it is in hang status.
>
> I see running but it seems hanging, no log from 1 hour
>
>
>
> This is the last manifoldcf.log lines:
>
>
>
>
>
>         at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446)
> [mcf-jcifs-connector.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222)
> [mcf-jcifs-connector.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-28T14:46:21,524 (Worker thread '59') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
>         at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446)
> [mcf-jcifs-connector.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222)
> [mcf-jcifs-connector.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-28T14:46:25,232 (Worker thread '45') - Tika Server: Tika
> Server rejects: Tika Server rejected document with the following reason:
> Unprocessable Entity
>
> WARN 2018-11-28T14:46:25,277 (Worker thread '70') - Tika Server: Tika
> Server rejects: Tika Server rejected document with the following reason:
> Unprocessable Entity
>
> WARN 2018-11-28T14:46:28,825 (Worker thread '27') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
>         at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446)
> [mcf-jcifs-connector.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222)
> [mcf-jcifs-connector.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-28T14:46:40,473 (Worker thread '79') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
>         at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446)
> [mcf-jcifs-connector.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222)
> [mcf-jcifs-connector.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-28T14:46:54,855 (Worker thread '65') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
>         at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446)
> [mcf-jcifs-connector.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222)
> [mcf-jcifs-connector.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-11-28T14:46:55,175 (Worker thread '83') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
>         at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2446)
> [mcf-jcifs-connector.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1222)
> [mcf-jcifs-connector.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
>
>
>
>
>
>
> I don’t know what to check.
>
>
>
> Tika server is ok, and it doesn’t more restart it .
>
>
>
>
>
>
>
> *Da:* Karl Wright <daddywri@gmail.com>
> *Inviato:* martedì 6 novembre 2018 15:27
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Job stuck without message
>
>
>
> I added a couple of questions to the ticket.  Please reply.
>
>
>
> Thanks,
>
> Karl
>
>
>
>
>
> On Tue, Nov 6, 2018 at 8:56 AM Bisonti Mario <Mario.Bisonti@vimar.com>
> wrote:
>
> Thanks a lot, Karl.
>
> I created a ticket.
>
> https://issues.apache.org/jira/browse/CONNECTORS-1554
> <https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FCONNECTORS-1554&data=01%7C01%7CMario.Bisonti%40vimar.com%7C04a2b2ff508d4ac0254908d655473c51%7Ca1f008bcd59b4c668f8760fd9af15c7f%7C1&sdata=xFzROisXXTNNBkGZwOE3%2F4ypIROkxz82CgW%2BvS%2Bwqgw%3D&reserved=0>
>
>
>
>
>
> Thanks
>
>
>
> Mario
>
>
>
>
>
>
>
> *Da:* Karl Wright <daddywri@gmail.com>
> *Inviato:* martedì 6 novembre 2018 14:28
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Job stuck without message
>
>
>
> ok, can you create a ticket?  Also, I'd appreciate it if you can look at
> the simple history for one of these documents; I need to see what happened
> to it last.
>
>
>
> Thanks,
>
> Karl
>
>
>
>
>
> On Tue, Nov 6, 2018 at 7:32 AM Bisonti Mario <Mario.Bisonti@vimar.com>
> wrote:
>
> My version is 2.11
>
>
>
>
>
>
>
>
>
> *Da:* Karl Wright <daddywri@gmail.com>
> *Inviato:* martedì 6 novembre 2018 13:07
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Job stuck without message
>
>
>
> Thanks.
>
> What version of ManifoldCF are you using?  We fixed a problem a while back
> having to do with documents that (because of error processing) get put into
> a "ready for processing" state which don't have any document priority set.
> But this should have been addressed, certainly, by the most recent release
> and probably by 2.10 as well.
>
>
>
> Karl
>
>
>
>
>
> On Tue, Nov 6, 2018 at 5:43 AM Bisonti Mario <Mario.Bisonti@vimar.com>
> wrote:
>
> Hallo Karl.
>
> When it hangs I see in the Queue status:
>
>
>
> And in the Document Status:
>
>
>
>
>
> *Da:* Karl Wright <daddywri@gmail.com>
> *Inviato:* martedì 30 ottobre 2018 19:32
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Job stuck without message
>
>
>
> What I am interested in now is the Document Status report for any one of
> the documents that is 'stuck'.  The next crawl time value is the critical
> field.  Can you include an example?
>
>
>
> Karl
>
>
>
> On Tue, Oct 30, 2018, 12:36 PM Bisonti Mario <Mario.Bisonti@vimar.com>
> wrote:
>
> Thanks a lot, Karl.
>
>
>
> It happens that the job starts, it works and index for an hour and after
> it frezzes, I haven’t error or waiting status in Document Queue o Simple
> History, I have only “OK” status so, I haven’t failures.
>
>
>
> I am not able to see other log errors other from the manifoldcf.log
>
>
>
> Solr server is ok
>
> Tika server is ok
>
> Agent is ok
>
> Tomcat with ManifoldCF is ok
>
>
>
> I could search if I could to put in info log mode for example Tika servrer
> or Solr.
>
>
>
> Thanks..
>
>
>
>
>
> *Da:* Karl Wright <daddywri@gmail.com>
> *Inviato:* martedì 30 ottobre 2018 16:38
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Job stuck without message
>
>
>
> Hi Mario,
>
> Please look at the Queue Status report to determine what is waiting and
> why it is waiting.
> You can also look at the Simple History to see what has been happening.
> If you are getting 100% failures in fetching documents then you may need to
> address this because your infrastructure is unhappy.  If the failure is
> something that indicates that the document is never going to be readable,
> that's a different problem and we might need to address that in the
> connector.
>
>
>
> Karl
>
>
>
>
>
> On Tue, Oct 30, 2018 at 10:33 AM Bisonti Mario <Mario.Bisonti@vimar.com>
> wrote:
>
>
>
> Thanks a lot Karl
>
>
>
> Yes, I see many docs in the docs queue but they are inactive.
>
>
>
> Infact i see that no more docs are indexed in Solr and I see that job is
> with the same number of docs Active (35012)
>
>
>
>
>
>
>
>
>
> *Da:* Karl Wright <daddywri@gmail.com>
> *Inviato:* martedì 30 ottobre 2018 13:59
> *A:* user@manifoldcf.apache.org
> *Oggetto:* Re: Job stuck without message
>
>
>
> The reason the job is "stuck" is because:
>
> ' JCIFS: Possibly transient exception detected on attempt 1 while getting
> share security: All pipe instances are busy.'
>
> This means that ManifoldCF will retry this document for a while before it
> gives up on it.  It appears to be stuck but it is not.  You can verify that
> by looking at the Document Queue report to see what is queued and what
> times the various documents will be retried.
>
>
>
> Karl
>
>
>
>
>
> On Tue, Oct 30, 2018 at 5:07 AM Bisonti Mario <Mario.Bisonti@vimar.com>
> wrote:
>
> Hallo.
>
>
>
> I started a job that works for some minutes, and after it stucks.
>
>
>
> In the manifoldcf.log I see:
>         at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-10-30T09:21:31,440 (Worker thread '2') - Tika Server: Tika
> Server rejects: Tika Server rejected document with the following reason:
> Unprocessable Entity
>
> WARN 2018-10-30T09:21:33,502 (Worker thread '14') - Tika Server: Tika
> Server rejects: Tika Server rejected document with the following reason:
> Unprocessable Entity
>
> WARN 2018-10-30T09:21:37,725 (Worker thread '30') - Tika Server: Tika
> Server rejects: Tika Server rejected document with the following reason:
> Unprocessable Entity
>
> WARN 2018-10-30T09:21:44,406 (Worker thread '49') - Tika Server: Tika
> Server rejects: Tika Server rejected document with the following reason:
> Unprocessable Entity
>
> WARN 2018-10-30T09:21:47,310 (Worker thread '15') - Tika Server: Tika
> Server rejects: Tika Server rejected document with the following reason:
> Unprocessable Entity
>
> WARN 2018-10-30T09:21:52,000 (Worker thread '27') - Tika Server: Tika
> Server rejects: Tika Server rejected document with the following reason:
> Unprocessable Entity
>
> WARN 2018-10-30T09:21:53,526 (Worker thread '15') - Tika Server: Tika
> Server rejects: Tika Server rejected document with the following reason:
> Unprocessable Entity
>
> WARN 2018-10-30T09:22:04,511 (Worker thread '3') - JCIFS: Possibly
> transient exception detected on attempt 1 while getting share security: All
> pipe instances are busy.
>
> jcifs.smb.SmbException: All pipe instances are busy.
>
>         at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbTransport.send(SmbTransport.java:669)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbSession.send(SmbSession.java:238)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.open0(SmbFile.java:993)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.open(SmbFile.java:1010)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951)
> ~[jcifs-1.3.18.3.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
> [mcf-jcifs-connector.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
> [mcf-jcifs-connector.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
> [mcf-jcifs-connector.jar:?]
>
>         at
> org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399)
> [mcf-pull-agent.jar:?]
>
> WARN 2018-10-30T09:22:10,359 (Worker thread '27') - Tika Server: Tika
> Server rejects: Tika Server rejected document with the following reason:
> Unprocessable Entity
>
> WARN 2018-10-30T09:22:13,932 (Worker thread '12') - Tika Server: Tika
> Server rejects: Tika Server rejected document with the following reason:
> Unprocessable Entity
>
> WARN 2018-10-30T09:22:14,274 (Worker thread '23') - Tika Server: Tika
> Server rejects: Tika Server rejected document with the following reason:
> Unprocessable Entity
>
> WARN 2018-10-30T09:22:19,933 (Worker thread '8') - Tika Server: Tika
> Server rejects: Tika Server rejected document with the following reason:
> Unprocessable Entity
>
> WARN 2018-10-30T09:23:59,920 (Worker thread '39') - Tika Server: Tika
> Server rejects: Tika Server rejected document with the following reason:
> Unprocessable Entity
>
> WARN 2018-10-30T09:24:09,059 (Worker thread '43') - Tika Server: Tika
> Server rejects: Tika Server rejected document with the following reason:
> Unprocessable Entity
>
>
>
>
>
>
>
> What could I check?
>
>
>
> Tika server works as standalone.
>
>
>
> Could you help  me?
>
>
>
> Thanks a lot
>
>
>
> Mario
>
>
>
>

Mime
View raw message