lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin Grotzke <martin.grot...@googlemail.com>
Subject Solrj retry handling - prevent "ProtocolException: Unbuffered entity enclosing request can not be repeated"
Date Tue, 12 Apr 2011 11:52:29 GMT
Hi,

from time to time we're seeing a "ProtocolException: Unbuffered entity
enclosing request can not be repeated." in the logs when sending ~500
docs to solr (the stack trace is at the end of the email).

I'm aware that this was discussed before (e.g. [1]) and our solution was
already to reduce the number of docs that are sent to solr.

However, I think that the issue might be solved in solrj. This
discussion on the httpclient-dev mailing list [2] points out the
solution under option "3) re-instantiate the input stream and retry the
request manually".

AFAICS CommonsHttpSolrServer.request when _maxRetries is set to s.th. >
0 (see [3]) already does some retry stuff, but not around the actual
http method execution (_httpClient.executeMethod(method)). Not sure for
what the several tries are implemented, but I'd say that if the user
sets maxRetries to s.th. > 0 also http method execution should be retried.

Another thing is the actually seen ProtocolException: AFAICS this is
thrown as httpclient (HttpMethodDirector.executeWithRetry) performs a
retry itself (see [4]) while the actually processed HttpMethod does not
support this.

As HttpMethodDirector.executeWithRetry already checks for a
HttpMethodRetryHandler (under param HttpMethodParams.RETRY_HANDLER,
[5]), it seems as if it would be enough to add such a handler for the
update/POST requests to prevent the ProtocolException.

So in summary I suggest two things:
1) Retry http method execution when maxRetiries is > 0
2) Prevent HttpClient from doing retries (by adding HttpMethodRetryHandler)

I first wanted to post it here on the list to see if there are
objections or other solutions. Or if there are plans to replace commons
httpclient (3.x) by s.th. like apache httpclient 4.x or async-http-client.

If there's an agreement that the proposed solution is the way to go ATM
I'd submit an appropriate issue for this.

Any comments?

Cheers,
Martin



[1]
http://lucene.472066.n3.nabble.com/Unbuffered-entity-enclosing-request-can-not-be-repeated-tt788186.html

[2]
http://www.mail-archive.com/commons-httpclient-dev@jakarta.apache.org/msg06723.html

[3]
http://svn.apache.org/viewvc/lucene/dev/trunk/solr/src/solrj/org/apache/solr/client/solrj/impl/CommonsHttpSolrServer.java?view=markup#l281

[4]
http://svn.apache.org/viewvc/httpcomponents/oac.hc3x/trunk/src/java/org/apache/commons/httpclient/HttpMethodDirector.java?view=markup#l366

[5]
http://svn.apache.org/viewvc/httpcomponents/oac.hc3x/trunk/src/java/org/apache/commons/httpclient/HttpMethodDirector.java?view=markup#l426


Stack trace:

Caused by: org.apache.commons.httpclient.ProtocolException: Unbuffered
entity enclosing request can not be repeated.
        at
org.apache.commons.httpclient.methods.EntityEnclosingMethod.writeRequestBody(EntityEnclosingMethod.java:487)
        at
org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:2110)
        at
org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1088)
        at
org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
        at
org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
        at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
        at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
        at
org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:427)


-- 
Martin Grotzke
http://twitter.com/martin_grotzke


Mime
View raw message