lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christopher Baird" <cba...@cardinalcommerce.com>
Subject ContentStreamUpdateRequest - out of memory on a large file
Date Wed, 12 May 2010 17:58:30 GMT
We're running into an out of memory problem when sending a large file to our
SOLR server using the ContentStreamUpdateRequest.  It appears that this
happens because when the request method of CommonsHttpSolrServer is called
(this is called even when using a StreamingUpdateSolrServer instance because
the ContentStreamUpdateRequest class is not an instance of UpdateRequest) an
InputStreamRequestEntity is used in the PostMethod buffers the content.  The
buffering happens because the content length is not provided and thus
defaults to "CONTENT_LENGHT_AUTO" which instructs InputStreamRequestEntity
to buffer the entire content.

 

Is there an existing work-around to this?

 

If not, can anyone think of why I wouldn't want to update the code to pass
in the content-length and avoid the buffering (I don't want to walk down a
path to find out I really stepped in something).

 

Thanks

-Chris

 

--------------------------------------------------

Christopher Baird

CardinalCommerce Corporation

+1.440.352.8444 ext 114 (Office)

+1.440.289.2266 (Mobile)

e-mail cbaird@cardinalcommerce.com
<blocked::mailto:cbaird@cardinalcommerce.com> 

 

THIS MESSAGE IS INTENDED ONLY FOR THE USE OF THE INDIVIDUAL OR ENTITY TO
WHICH IT IS ADDRESSED AND CONTAINS INFORMATION THAT IS PRIVILEGED,
CONFIDENTIAL, AND EXEMPT FROM DISCLOSURE UNDER APPLICABLE LAW.

If the reader of this message is not the intended recipient, or the employee
or agent responsible
for delivering the message to the intended recipient, you are hereby
notified that any dissemination,
distribution, forwarding, or copying of this communication is strictly
prohibited. If you have received
this communication in error, please notify the sender immediately by e-mail
or telephone, and delete
the original message immediately.

 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message