manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: IO exception during indexing: null
Date Mon, 11 Mar 2013 14:27:35 GMT
Hi Erlend,

The more I look at this, and the more I hear, the more I believe a
serious bug has crept into HttpClient.  At the very least it appears
that expect-continue is not working properly.  I think that the
HttpClient team turned off expect-continue by default because of
compatibility issues with older proxies (that is what they told me),
and there's a good chance that was why you needed to change your code.
 However , it is now apparently broken so badly that expect-continue
doesn't work properly at all - for the very case it was designed for.

I am an httpclient committer, so if we can show what the problem is, I
can propose a fix and commit it.  Let's therefore do the research ASAP
so we can get this problem fixed before the next 4.2.x release.

Thanks,
Karl

On Mon, Mar 11, 2013 at 10:05 AM, Karl Wright <daddywri@gmail.com> wrote:
> We already use multipart entity.  But we can't use bytearraybody
> because it means the entire document has to be sucked into memory.
> That's not acceptable.
>
> I will post again on the HttpClient list, because they did lead me to
> believe that using expect-continue would do the right thing.
>
> Karl
>
>
> On Mon, Mar 11, 2013 at 9:58 AM, Erlend Garåsen <e.f.garasen@usit.uio.no> wrote:
>>
>> Did an svn up, built and deployed on our prod server, but I'm still getting
>> the same error.
>>
>> I just saw the following comment from you in the HttpClient forum: "Problem
>> is that ManifoldCF output connectors get an inputstream handed to them, not
>> a file.".
>>
>> I think I had a similar issue when I upgraded from a previous release of
>> HttpClient. By using the HttpPost class in HttpClient, I did something like
>> this with a dummy file named used (for multi-form post data):
>>
>> MultipartEntity multipartEntity = new MultipartEntity();
>> multipartEntity.addPart("myfile", new ByteArrayBody(content, contentType,
>> "myfile"));
>> httpPost.setEntity(multipartEntity);
>>
>> I'm afraid that we need to use the new HttpEntity implementations.
>>
>> Erlend
>>
>>
>> On 10.03.13 20.43, Erlend Garåsen wrote:
>>>
>>> Thanks Karl!
>>>
>>> I'm sitting on the train this evening, so I will try and report tomorrow.
>>>
>>> Erlend.
>>>
>>> Sent from my iPad
>>>
>>> On 8. mars 2013, at 21:06, Karl Wright <daddywri@gmail.com> wrote:
>>>
>>>> There's some indication that turning on expect-continue will resolve
>>>> this issue.  So that is what I did.  Please synch up trunk and see if
>>>> you still get the error.
>>>>
>>>> Thanks!
>>>> Karl
>>>>
>>>>
>>>> On Fri, Mar 8, 2013 at 9:14 AM, Karl Wright <daddywri@gmail.com> wrote:
>>>>>
>>>>> OK, there IS a problem here, and it is a significant one.  See
>>>>> CONNECTORS-661 for details.
>>>>>
>>>>> I've asked for a workaround proposal on the httpcomponents list; it
>>>>> seems to me that pre-emptive auth headers should be possible in
>>>>> HttpClient.  In the worst case, we can construct our own header, but
>>>>> first let's see what they say.
>>>>>
>>>>> Karl
>>>>>
>>>>>
>>>>> On Fri, Mar 8, 2013 at 8:20 AM, Erlend Garåsen <e.f.garasen@usit.uio.no>
>>>>> wrote:
>>>>>>
>>>>>> On 08.03.13 13.37, Karl Wright wrote:
>>>>>>>
>>>>>>>
>>>>>>> Actually, let's just cut to the chase, and enable full wire and
>>>>>>> context
>>>>>>> logging:
>>>>>>>
>>>>>>> log4j.logger.org.apache.http=DEBUG
>>>>>>>
>>>>>>> That will let us see what is happening for sure.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Working on it.
>>>>>>
>>>>>> I will send the log directly to you, Karl, since this list may cut
the
>>>>>> log
>>>>>> file.
>>>>>>
>>>>>>
>>>>>> Erlend
>>>>>>
>>>>>> --
>>>>>> Erlend Garåsen
>>>>>> Center for Information Technology Services
>>>>>> University of Oslo
>>>>>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>>>>>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968,
VIP:
>>>>>> 31050
>>
>>
>>
>> --
>> Erlend Garåsen
>> Center for Information Technology Services
>> University of Oslo
>> P.O. Box 1086 Blindern, N-0317 OSLO, Norway
>> Ph: (+47) 22840193, Fax: (+47) 22852970, Mobile: (+47) 91380968, VIP: 31050

Mime
View raw message