manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bisonti Mario <Mario.Biso...@vimar.com>
Subject R: Job stuck without message
Date Tue, 06 Nov 2018 13:56:31 GMT
Thanks a lot, Karl.
I created a ticket.
https://issues.apache.org/jira/browse/CONNECTORS-1554


Thanks

Mario



Da: Karl Wright <daddywri@gmail.com>
Inviato: martedì 6 novembre 2018 14:28
A: user@manifoldcf.apache.org
Oggetto: Re: Job stuck without message

ok, can you create a ticket?  Also, I'd appreciate it if you can look at the simple history
for one of these documents; I need to see what happened to it last.

Thanks,
Karl


On Tue, Nov 6, 2018 at 7:32 AM Bisonti Mario <Mario.Bisonti@vimar.com<mailto:Mario.Bisonti@vimar.com>>
wrote:
My version is 2.11




Da: Karl Wright <daddywri@gmail.com<mailto:daddywri@gmail.com>>
Inviato: martedì 6 novembre 2018 13:07
A: user@manifoldcf.apache.org<mailto:user@manifoldcf.apache.org>
Oggetto: Re: Job stuck without message

Thanks.
What version of ManifoldCF are you using?  We fixed a problem a while back having to do with
documents that (because of error processing) get put into a "ready for processing" state which
don't have any document priority set.  But this should have been addressed, certainly, by
the most recent release and probably by 2.10 as well.

Karl


On Tue, Nov 6, 2018 at 5:43 AM Bisonti Mario <Mario.Bisonti@vimar.com<mailto:Mario.Bisonti@vimar.com>>
wrote:
Hallo Karl.
When it hangs I see in the Queue status:

And in the Document Status:


Da: Karl Wright <daddywri@gmail.com<mailto:daddywri@gmail.com>>
Inviato: martedì 30 ottobre 2018 19:32
A: user@manifoldcf.apache.org<mailto:user@manifoldcf.apache.org>
Oggetto: Re: Job stuck without message

What I am interested in now is the Document Status report for any one of the documents that
is 'stuck'.  The next crawl time value is the critical field.  Can you include an example?

Karl

On Tue, Oct 30, 2018, 12:36 PM Bisonti Mario <Mario.Bisonti@vimar.com<mailto:Mario.Bisonti@vimar.com>>
wrote:
Thanks a lot, Karl.

It happens that the job starts, it works and index for an hour and after it frezzes, I haven’t
error or waiting status in Document Queue o Simple History, I have only “OK” status so,
I haven’t failures.

I am not able to see other log errors other from the manifoldcf.log

Solr server is ok
Tika server is ok
Agent is ok
Tomcat with ManifoldCF is ok

I could search if I could to put in info log mode for example Tika servrer or Solr.

Thanks..


Da: Karl Wright <daddywri@gmail.com<mailto:daddywri@gmail.com>>
Inviato: martedì 30 ottobre 2018 16:38
A: user@manifoldcf.apache.org<mailto:user@manifoldcf.apache.org>
Oggetto: Re: Job stuck without message

Hi Mario,

Please look at the Queue Status report to determine what is waiting and why it is waiting.
You can also look at the Simple History to see what has been happening.  If you are getting
100% failures in fetching documents then you may need to address this because your infrastructure
is unhappy.  If the failure is something that indicates that the document is never going to
be readable, that's a different problem and we might need to address that in the connector.

Karl


On Tue, Oct 30, 2018 at 10:33 AM Bisonti Mario <Mario.Bisonti@vimar.com<mailto:Mario.Bisonti@vimar.com>>
wrote:

Thanks a lot Karl

Yes, I see many docs in the docs queue but they are inactive.

Infact i see that no more docs are indexed in Solr and I see that job is with the same number
of docs Active (35012)




Da: Karl Wright <daddywri@gmail.com<mailto:daddywri@gmail.com>>
Inviato: martedì 30 ottobre 2018 13:59
A: user@manifoldcf.apache.org<mailto:user@manifoldcf.apache.org>
Oggetto: Re: Job stuck without message

The reason the job is "stuck" is because:

' JCIFS: Possibly transient exception detected on attempt 1 while getting share security:
All pipe instances are busy.'

This means that ManifoldCF will retry this document for a while before it gives up on it.
 It appears to be stuck but it is not.  You can verify that by looking at the Document Queue
report to see what is queued and what times the various documents will be retried.

Karl


On Tue, Oct 30, 2018 at 5:07 AM Bisonti Mario <Mario.Bisonti@vimar.com<mailto:Mario.Bisonti@vimar.com>>
wrote:
Hallo.

I started a job that works for some minutes, and after it stucks.

In the manifoldcf.log I see:
        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
[mcf-jcifs-connector.jar:?]
        at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-10-30T09:21:31,440 (Worker thread '2') - Tika Server: Tika Server rejects: Tika
Server rejected document with the following reason: Unprocessable Entity
WARN 2018-10-30T09:21:33,502 (Worker thread '14') - Tika Server: Tika Server rejects: Tika
Server rejected document with the following reason: Unprocessable Entity
WARN 2018-10-30T09:21:37,725 (Worker thread '30') - Tika Server: Tika Server rejects: Tika
Server rejected document with the following reason: Unprocessable Entity
WARN 2018-10-30T09:21:44,406 (Worker thread '49') - Tika Server: Tika Server rejects: Tika
Server rejected document with the following reason: Unprocessable Entity
WARN 2018-10-30T09:21:47,310 (Worker thread '15') - Tika Server: Tika Server rejects: Tika
Server rejected document with the following reason: Unprocessable Entity
WARN 2018-10-30T09:21:52,000 (Worker thread '27') - Tika Server: Tika Server rejects: Tika
Server rejected document with the following reason: Unprocessable Entity
WARN 2018-10-30T09:21:53,526 (Worker thread '15') - Tika Server: Tika Server rejects: Tika
Server rejected document with the following reason: Unprocessable Entity
WARN 2018-10-30T09:22:04,511 (Worker thread '3') - JCIFS: Possibly transient exception detected
on attempt 1 while getting share security: All pipe instances are busy.
jcifs.smb.SmbException: All pipe instances are busy.
        at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:569) ~[jcifs-1.3.18.3.jar:?]
        at jcifs.smb.SmbTransport.send(SmbTransport.java:669) ~[jcifs-1.3.18.3.jar:?]
        at jcifs.smb.SmbSession.send(SmbSession.java:238) ~[jcifs-1.3.18.3.jar:?]
        at jcifs.smb.SmbTree.send(SmbTree.java:119) ~[jcifs-1.3.18.3.jar:?]
        at jcifs.smb.SmbFile.send(SmbFile.java:776) ~[jcifs-1.3.18.3.jar:?]
        at jcifs.smb.SmbFile.open0(SmbFile.java:993) ~[jcifs-1.3.18.3.jar:?]
        at jcifs.smb.SmbFile.open(SmbFile.java:1010) ~[jcifs-1.3.18.3.jar:?]
        at jcifs.smb.SmbFileOutputStream.<init>(SmbFileOutputStream.java:142) ~[jcifs-1.3.18.3.jar:?]
        at jcifs.smb.TransactNamedPipeOutputStream.<init>(TransactNamedPipeOutputStream.java:32)
~[jcifs-1.3.18.3.jar:?]
        at jcifs.smb.SmbNamedPipe.getNamedPipeOutputStream(SmbNamedPipe.java:187) ~[jcifs-1.3.18.3.jar:?]
        at jcifs.dcerpc.DcerpcPipeHandle.doSendFragment(DcerpcPipeHandle.java:68) ~[jcifs-1.3.18.3.jar:?]
        at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:190) ~[jcifs-1.3.18.3.jar:?]
        at jcifs.dcerpc.DcerpcHandle.bind(DcerpcHandle.java:126) ~[jcifs-1.3.18.3.jar:?]
        at jcifs.dcerpc.DcerpcHandle.sendrecv(DcerpcHandle.java:140) ~[jcifs-1.3.18.3.jar:?]
        at jcifs.smb.SmbFile.getShareSecurity(SmbFile.java:2951) ~[jcifs-1.3.18.3.jar:?]
        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecurity(SharedDriveConnector.java:2438)
[mcf-jcifs-connector.jar:?]
        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.getFileShareSecuritySet(SharedDriveConnector.java:1221)
[mcf-jcifs-connector.jar:?]
        at org.apache.manifoldcf.crawler.connectors.sharedrive.SharedDriveConnector.processDocuments(SharedDriveConnector.java:627)
[mcf-jcifs-connector.jar:?]
        at org.apache.manifoldcf.crawler.system.WorkerThread.run(WorkerThread.java:399) [mcf-pull-agent.jar:?]
WARN 2018-10-30T09:22:10,359 (Worker thread '27') - Tika Server: Tika Server rejects: Tika
Server rejected document with the following reason: Unprocessable Entity
WARN 2018-10-30T09:22:13,932 (Worker thread '12') - Tika Server: Tika Server rejects: Tika
Server rejected document with the following reason: Unprocessable Entity
WARN 2018-10-30T09:22:14,274 (Worker thread '23') - Tika Server: Tika Server rejects: Tika
Server rejected document with the following reason: Unprocessable Entity
WARN 2018-10-30T09:22:19,933 (Worker thread '8') - Tika Server: Tika Server rejects: Tika
Server rejected document with the following reason: Unprocessable Entity
WARN 2018-10-30T09:23:59,920 (Worker thread '39') - Tika Server: Tika Server rejects: Tika
Server rejected document with the following reason: Unprocessable Entity
WARN 2018-10-30T09:24:09,059 (Worker thread '43') - Tika Server: Tika Server rejects: Tika
Server rejected document with the following reason: Unprocessable Entity



What could I check?

Tika server works as standalone.

Could you help  me?

Thanks a lot

Mario

Mime
View raw message