manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: SharePoint crawler ArrayIndexOutOfBoundException in log
Date Fri, 17 Mar 2017 14:28:29 GMT
Hi Cihad,

Could you also check out and build the latest 4.5.x httpclient, from this
branch?

https://svn.apache.org/repos/asf/httpcomponents/httpclient/branches/pull-66

You will need maven for this but otherwise you can build it any way you
like.  Replace the "httpclient-4.5.1.jar" in the lib directory with the jar
you build, and then you can rebuild MCF.  See if you still get the error.
If you do, it should be possible to chase it down more readily.

Thanks,
Karl


On Fri, Mar 17, 2017 at 9:57 AM, Cihad Guzel <cguzelg@gmail.com> wrote:

> No. I don't use any custom library.
>
> I try with manifoldcf trunk on my notebook. I install sharepoint 2013 on
> ms server 2012 for testing with default configuration.
>
> 17 Mar 2017 16:05 tarihinde "Karl Wright" <daddywri@gmail.com> yazdı:
>
> Hmm, I can see no way this can happen.  Are you by any chance using a
>> modified version of the HttpClient library?
>> Karl
>>
>>
>> On Fri, Mar 17, 2017 at 8:09 AM, Karl Wright <daddywri@gmail.com> wrote:
>>
>>> Hi Cihad,
>>>
>>> This is very interesting because the problem is coming from Httpclient's
>>> NTLM engine.  The allocated packet size for the Type 1 message is being
>>> exceeded, which I didn't think was even possible.
>>>
>>> This may be a result of credentials that you have supplied being strange
>>> in some way.  Let me look at the Httpclient code and get back to you.
>>>
>>> Karl
>>>
>>>
>>> On Fri, Mar 17, 2017 at 7:57 AM, Cihad Guzel <cguzelg@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I try sharepoint connector with Active Directory in debug mode. I saw
>>>> ArrayIndexOutOfBoundException in manifoldcf.log file. Any bugs?
>>>>
>>>> DEBUG 2017-03-17 14:30:48,386 (Worker thread '0') - SharePoint: Getting
>>>> version of '/Documents2//Step by step Installation of SharePoint 2013 on
>>>> Windows Server 2012 R2 part 1 - SharePoint Community.pdf'
>>>> DEBUG 2017-03-17 14:30:48,466 (Worker thread '0') - SharePoint:
>>>> Checking whether to include document '/Documents2/Step by step Installation
>>>> of SharePoint 2013 on Windows Server 2012 R2 part 1 - SharePoint
>>>> Community.pdf'
>>>> DEBUG 2017-03-17 14:30:48,466 (Worker thread '0') - SharePoint: File
>>>> '/Documents2/Step by step Installation of SharePoint 2013 on Windows Server
>>>> 2012 R2 part 1 - SharePoint Community.pdf' exactly matched rule path
>>>> '/Documents2/*'
>>>> DEBUG 2017-03-17 14:30:48,467 (Worker thread '0') - SharePoint:
>>>> Including file '/Documents2/Step by step Installation of SharePoint 2013
on
>>>> Windows Server 2012 R2 part 1 - SharePoint Community.pdf'
>>>> DEBUG 2017-03-17 14:30:48,468 (Worker thread '0') - SharePoint: Finding
>>>> metadata to include for document/item '/Documents2/Step by step
>>>> Installation of SharePoint 2013 on Windows Server 2012 R2 part 1 -
>>>> SharePoint Community.pdf'.
>>>> DEBUG 2017-03-17 14:30:48,510 (Worker thread '0') - SharePoint: In
>>>> getFieldValues; fieldNames=[Ljava.lang.String;@69f1a61a, site='',
>>>> docLibrary='{1B694C45-DF1F-44E7-9814-F5096E85A126}',
>>>> docId='/Documents2/Step by step Installation of SharePoint 2013 on Windows
>>>> Server 2012 R2 part 1 - SharePoint Community.pdf', dspStsWorks=false
>>>> DEBUG 2017-03-17 14:30:48,539 (Worker thread '5') - SharePoint: Getting
>>>> version of '/Documents2//'
>>>> DEBUG 2017-03-17 14:30:48,539 (Worker thread '4') - SharePoint: Getting
>>>> version of '/Documents2//CXFCA3100080010.pdf'
>>>> DEBUG 2017-03-17 14:30:48,539 (Worker thread '4') - SharePoint:
>>>> Checking whether to include document '/Documents2/CXFCA3100080010.pdf'
>>>> DEBUG 2017-03-17 14:30:48,539 (Worker thread '4') - SharePoint: File
>>>> '/Documents2/CXFCA3100080010.pdf' exactly matched rule path
>>>> '/Documents2/*'
>>>> DEBUG 2017-03-17 14:30:48,539 (Worker thread '4') - SharePoint:
>>>> Including file '/Documents2/CXFCA3100080010.pdf'
>>>> DEBUG 2017-03-17 14:30:48,539 (Worker thread '5') - SharePoint:
>>>> Checking whether to include library '/Documents2'
>>>> DEBUG 2017-03-17 14:30:48,539 (Worker thread '4') - SharePoint: Finding
>>>> metadata to include for document/item '/Documents2/CXFCA3100080010.p
>>>> df'.
>>>> DEBUG 2017-03-17 14:30:48,539 (Worker thread '5') - SharePoint: Library
>>>> '/Documents2' partially matched file rule path '/Documents2/*' - including
>>>> DEBUG 2017-03-17 14:30:48,539 (Worker thread '5') - SharePoint:
>>>> Document identifier is a library: '/Documents2'
>>>> DEBUG 2017-03-17 14:30:48,539 (Worker thread '5') - SharePoint: In
>>>> getDocLibID; parentSite='', parentSiteDecoded='', docLibrary='Documents2'
>>>> DEBUG 2017-03-17 14:30:48,540 (Worker thread '2') - SharePoint: Getting
>>>> version of '/'
>>>> DEBUG 2017-03-17 14:30:48,540 (Worker thread '2') - SharePoint:
>>>> Checking whether to include site '/'
>>>> DEBUG 2017-03-17 14:30:48,540 (Worker thread '2') - SharePoint: Site
>>>> '/' partially matched file rule path '/Documents2/*' - including
>>>> DEBUG 2017-03-17 14:30:48,548 (Worker thread '4') - SharePoint: In
>>>> getFieldValues; fieldNames=[Ljava.lang.String;@6f447d2e, site='',
>>>> docLibrary='{1B694C45-DF1F-44E7-9814-F5096E85A126}',
>>>> docId='/Documents2/CXFCA3100080010.pdf', dspStsWorks=false
>>>> DEBUG 2017-03-17 14:30:48,560 (Worker thread '2') - SharePoint:
>>>> Document identifier is a site: ''
>>>> DEBUG 2017-03-17 14:30:48,560 (Worker thread '2') - SharePoint: In
>>>> getSites; parentSite=''
>>>> DEBUG 2017-03-17 14:30:50,398 (Worker thread '4') - SharePoint: Got a
>>>> remote exception getting field values for site  library
>>>> {1B694C45-DF1F-44E7-9814-F5096E85A126} document
>>>> [/Documents2/CXFCA3100080010.pdf] - retrying
>>>> AxisFault
>>>>  faultCode: {http://schemas.xmlsoap.org/so
>>>> ap/envelope/}Server.userException
>>>>  faultSubcode:
>>>>  faultString: java.lang.ArrayIndexOutOfBoundsException: 41
>>>>  faultActor:
>>>>  faultNode:
>>>>  faultDetail:
>>>> {http://xml.apache.org/axis/}stackTrace:java.lang.ArrayIndex
>>>> OutOfBoundsException: 41
>>>> at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addByte
>>>> (NTLMEngineImpl.java:911)
>>>> at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addULon
>>>> g(NTLMEngineImpl.java:941)
>>>> at org.apache.http.impl.auth.NTLMEngineImpl$Type1Message.getRes
>>>> ponse(NTLMEngineImpl.java:1043)
>>>> at org.apache.http.impl.auth.NTLMEngineImpl.getType1Message(NTL
>>>> MEngineImpl.java:148)
>>>> at org.apache.http.impl.auth.NTLMEngineImpl.generateType1Msg(NT
>>>> LMEngineImpl.java:1628)
>>>> at org.apache.http.impl.auth.NTLMScheme.authenticate(NTLMScheme
>>>> .java:139)
>>>> at org.apache.http.impl.auth.AuthSchemeBase.authenticate(AuthSc
>>>> hemeBase.java:138)
>>>> at org.apache.http.impl.auth.HttpAuthenticator.doAuth(HttpAuthe
>>>> nticator.java:239)
>>>> at org.apache.http.impl.auth.HttpAuthenticator.generateAuthResp
>>>> onse(HttpAuthenticator.java:202)
>>>> at org.apache.http.impl.execchain.MainClientExec.execute(MainCl
>>>> ientExec.java:262)
>>>> at org.apache.http.impl.execchain.ProtocolExec.execute(Protocol
>>>> Exec.java:184)
>>>> at org.apache.http.impl.execchain.RedirectExec.execute(Redirect
>>>> Exec.java:110)
>>>> at org.apache.http.impl.client.InternalHttpClient.doExecute(Int
>>>> ernalHttpClient.java:184)
>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos
>>>> eableHttpClient.java:82)
>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos
>>>> eableHttpClient.java:107)
>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos
>>>> eableHttpClient.java:55)
>>>> at org.apache.manifoldcf.connectorcommon.common.CommonsHTTPSend
>>>> er$BackgroundHTTPThread.run(CommonsHTTPSender.java:654)
>>>>
>>>> {http://xml.apache.org/axis/}hostname:cg-DELL
>>>>
>>>> java.lang.ArrayIndexOutOfBoundsException: 41
>>>> at org.apache.axis.AxisFault.makeFault(AxisFault.java:101)
>>>> at org.apache.manifoldcf.connectorcommon.common.CommonsHTTPSend
>>>> er.invoke(CommonsHTTPSender.java:263)
>>>> at org.apache.axis.strategies.InvocationStrategy.visit(Invocati
>>>> onStrategy.java:32)
>>>> at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
>>>> at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
>>>> at org.apache.axis.client.AxisClient.invoke(AxisClient.java:165)
>>>> at org.apache.axis.client.Call.invokeEngine(Call.java:2784)
>>>> at org.apache.axis.client.Call.invoke(Call.java:2767)
>>>> at org.apache.axis.client.Call.invoke(Call.java:2443)
>>>> at org.apache.axis.client.Call.invoke(Call.java:2366)
>>>> at org.apache.axis.client.Call.invoke(Call.java:1812)
>>>> at com.microsoft.schemas.sharepoint.soap.ListsSoapStub.getListI
>>>> tems(ListsSoapStub.java:1841)
>>>> at org.apache.manifoldcf.crawler.connectors.sharepoint.SPSProxy
>>>> Helper.getFieldValues(SPSProxyHelper.java:2134)
>>>> at org.apache.manifoldcf.crawler.connectors.sharepoint.SharePoi
>>>> ntRepository.processDocuments(SharePointRepository.java:1433)
>>>> at org.apache.manifoldcf.crawler.system.WorkerThread.run(Worker
>>>> Thread.java:399)
>>>> Caused by: java.lang.ArrayIndexOutOfBoundsException: 41
>>>> at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addByte
>>>> (NTLMEngineImpl.java:911)
>>>> at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addULon
>>>> g(NTLMEngineImpl.java:941)
>>>> at org.apache.http.impl.auth.NTLMEngineImpl$Type1Message.getRes
>>>> ponse(NTLMEngineImpl.java:1043)
>>>> at org.apache.http.impl.auth.NTLMEngineImpl.getType1Message(NTL
>>>> MEngineImpl.java:148)
>>>> at org.apache.http.impl.auth.NTLMEngineImpl.generateType1Msg(NT
>>>> LMEngineImpl.java:1628)
>>>> at org.apache.http.impl.auth.NTLMScheme.authenticate(NTLMScheme
>>>> .java:139)
>>>> at org.apache.http.impl.auth.AuthSchemeBase.authenticate(AuthSc
>>>> hemeBase.java:138)
>>>> at org.apache.http.impl.auth.HttpAuthenticator.doAuth(HttpAuthe
>>>> nticator.java:239)
>>>> at org.apache.http.impl.auth.HttpAuthenticator.generateAuthResp
>>>> onse(HttpAuthenticator.java:202)
>>>> at org.apache.http.impl.execchain.MainClientExec.execute(MainCl
>>>> ientExec.java:262)
>>>> at org.apache.http.impl.execchain.ProtocolExec.execute(Protocol
>>>> Exec.java:184)
>>>> at org.apache.http.impl.execchain.RedirectExec.execute(Redirect
>>>> Exec.java:110)
>>>> at org.apache.http.impl.client.InternalHttpClient.doExecute(Int
>>>> ernalHttpClient.java:184)
>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos
>>>> eableHttpClient.java:82)
>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos
>>>> eableHttpClient.java:107)
>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos
>>>> eableHttpClient.java:55)
>>>> at org.apache.manifoldcf.connectorcommon.common.CommonsHTTPSend
>>>> er$BackgroundHTTPThread.run(CommonsHTTPSender.java:654)
>>>> DEBUG 2017-03-17 14:30:50,398 (Worker thread '0') - SharePoint: Got a
>>>> remote exception getting field values for site  library
>>>> {1B694C45-DF1F-44E7-9814-F5096E85A126} document [/Documents2/Step by
>>>> step Installation of SharePoint 2013 on Windows Server 2012 R2 part 1 -
>>>> SharePoint Community.pdf] - retrying
>>>> AxisFault
>>>>  faultCode: {http://schemas.xmlsoap.org/so
>>>> ap/envelope/}Server.userException
>>>>  faultSubcode:
>>>>  faultString: java.lang.ArrayIndexOutOfBoundsException: 41
>>>>  faultActor:
>>>>  faultNode:
>>>>  faultDetail:
>>>> {http://xml.apache.org/axis/}stackTrace:java.lang.ArrayIndex
>>>> OutOfBoundsException: 41
>>>> at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addByte
>>>> (NTLMEngineImpl.java:911)
>>>> at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addUSho
>>>> rt(NTLMEngineImpl.java:933)
>>>> at org.apache.http.impl.auth.NTLMEngineImpl$Type1Message.getRes
>>>> ponse(NTLMEngineImpl.java:1040)
>>>> at org.apache.http.impl.auth.NTLMEngineImpl.getType1Message(NTL
>>>> MEngineImpl.java:148)
>>>> at org.apache.http.impl.auth.NTLMEngineImpl.generateType1Msg(NT
>>>> LMEngineImpl.java:1628)
>>>> at org.apache.http.impl.auth.NTLMScheme.authenticate(NTLMScheme
>>>> .java:139)
>>>> at org.apache.http.impl.auth.AuthSchemeBase.authenticate(AuthSc
>>>> hemeBase.java:138)
>>>> at org.apache.http.impl.auth.HttpAuthenticator.doAuth(HttpAuthe
>>>> nticator.java:239)
>>>> at org.apache.http.impl.auth.HttpAuthenticator.generateAuthResp
>>>> onse(HttpAuthenticator.java:202)
>>>> at org.apache.http.impl.execchain.MainClientExec.execute(MainCl
>>>> ientExec.java:262)
>>>> at org.apache.http.impl.execchain.ProtocolExec.execute(Protocol
>>>> Exec.java:184)
>>>> at org.apache.http.impl.execchain.RedirectExec.execute(Redirect
>>>> Exec.java:110)
>>>> at org.apache.http.impl.client.InternalHttpClient.doExecute(Int
>>>> ernalHttpClient.java:184)
>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos
>>>> eableHttpClient.java:82)
>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos
>>>> eableHttpClient.java:107)
>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos
>>>> eableHttpClient.java:55)
>>>> at org.apache.manifoldcf.connectorcommon.common.CommonsHTTPSend
>>>> er$BackgroundHTTPThread.run(CommonsHTTPSender.java:654)
>>>>
>>>> {http://xml.apache.org/axis/}hostname:cg-DELL
>>>>
>>>> java.lang.ArrayIndexOutOfBoundsException: 41
>>>> at org.apache.axis.AxisFault.makeFault(AxisFault.java:101)
>>>> at org.apache.manifoldcf.connectorcommon.common.CommonsHTTPSend
>>>> er.invoke(CommonsHTTPSender.java:263)
>>>> at org.apache.axis.strategies.InvocationStrategy.visit(Invocati
>>>> onStrategy.java:32)
>>>> at org.apache.axis.SimpleChain.doVisiting(SimpleChain.java:118)
>>>> at org.apache.axis.SimpleChain.invoke(SimpleChain.java:83)
>>>> at org.apache.axis.client.AxisClient.invoke(AxisClient.java:165)
>>>> at org.apache.axis.client.Call.invokeEngine(Call.java:2784)
>>>> at org.apache.axis.client.Call.invoke(Call.java:2767)
>>>> at org.apache.axis.client.Call.invoke(Call.java:2443)
>>>> at org.apache.axis.client.Call.invoke(Call.java:2366)
>>>> at org.apache.axis.client.Call.invoke(Call.java:1812)
>>>> at com.microsoft.schemas.sharepoint.soap.ListsSoapStub.getListI
>>>> tems(ListsSoapStub.java:1841)
>>>> at org.apache.manifoldcf.crawler.connectors.sharepoint.SPSProxy
>>>> Helper.getFieldValues(SPSProxyHelper.java:2134)
>>>> at org.apache.manifoldcf.crawler.connectors.sharepoint.SharePoi
>>>> ntRepository.processDocuments(SharePointRepository.java:1433)
>>>> at org.apache.manifoldcf.crawler.system.WorkerThread.run(Worker
>>>> Thread.java:399)
>>>> Caused by: java.lang.ArrayIndexOutOfBoundsException: 41
>>>> at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addByte
>>>> (NTLMEngineImpl.java:911)
>>>> at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addUSho
>>>> rt(NTLMEngineImpl.java:933)
>>>> at org.apache.http.impl.auth.NTLMEngineImpl$Type1Message.getRes
>>>> ponse(NTLMEngineImpl.java:1040)
>>>> at org.apache.http.impl.auth.NTLMEngineImpl.getType1Message(NTL
>>>> MEngineImpl.java:148)
>>>> at org.apache.http.impl.auth.NTLMEngineImpl.generateType1Msg(NT
>>>> LMEngineImpl.java:1628)
>>>> at org.apache.http.impl.auth.NTLMScheme.authenticate(NTLMScheme
>>>> .java:139)
>>>> at org.apache.http.impl.auth.AuthSchemeBase.authenticate(AuthSc
>>>> hemeBase.java:138)
>>>> at org.apache.http.impl.auth.HttpAuthenticator.doAuth(HttpAuthe
>>>> nticator.java:239)
>>>> at org.apache.http.impl.auth.HttpAuthenticator.generateAuthResp
>>>> onse(HttpAuthenticator.java:202)
>>>> at org.apache.http.impl.execchain.MainClientExec.execute(MainCl
>>>> ientExec.java:262)
>>>> at org.apache.http.impl.execchain.ProtocolExec.execute(Protocol
>>>> Exec.java:184)
>>>> at org.apache.http.impl.execchain.RedirectExec.execute(Redirect
>>>> Exec.java:110)
>>>> at org.apache.http.impl.client.InternalHttpClient.doExecute(Int
>>>> ernalHttpClient.java:184)
>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos
>>>> eableHttpClient.java:82)
>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos
>>>> eableHttpClient.java:107)
>>>> at org.apache.http.impl.client.CloseableHttpClient.execute(Clos
>>>> eableHttpClient.java:55)
>>>> at org.apache.manifoldcf.connectorcommon.common.CommonsHTTPSend
>>>> er$BackgroundHTTPThread.run(CommonsHTTPSender.java:654)
>>>>  WARN 2017-03-17 14:30:50,404 (Worker thread '4') - Service
>>>> interruption reported for job 1489681799610 connection 'sp2013 crawl':
>>>> Remote procedure exception: ; nested exception is:
>>>> java.lang.ArrayIndexOutOfBoundsException: 41
>>>>  WARN 2017-03-17 14:30:50,405 (Worker thread '0') - Service
>>>> interruption reported for job 1489681799610 connection 'sp2013 crawl':
>>>> Remote procedure exception: ; nested exception is:
>>>> java.lang.ArrayIndexOutOfBoundsException: 41
>>>> DEBUG 2017-03-17 14:31:09,041 (Worker thread '2') - SharePoint:
>>>> getSites xml response: <ns1:Webs xmlns:ns1="http://schemas.micr
>>>> osoft.com/sharepoint/soap/"/>
>>>> DEBUG 2017-03-17 14:31:09,043 (Worker thread '2') - SharePoint: In
>>>> getDocumentLibraries; parentSite='', parentSiteDecoded=''
>>>>
>>>> --
>>>> Teşekkürler
>>>> Cihad Güzel
>>>>
>>>
>>>
>>

Mime
View raw message