lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hostetter <hossman_luc...@fucit.org>
Subject Re: Solrj ContentStreamUpdateRequest Slow
Date Mon, 16 Aug 2010 22:12:03 GMT

: > I think your problem may be that StreamingUpdateSolrServer buffers up
: > commands and sends them in batches in a background thread.  if you want to
: > send individual updates in real time (and time them) you should just use
: > CommonsHttpSolrServer
: 
: My goal is to batch updates.  My content lives somewhere else so I was trying
: to find a way to tell Solr where the document lived so it could go out and
: stream it into the index for me.  That's where I thought
: StreamingUpdateSolrServer would help.

If your content lives on a machine which is not your "client" nor your 
"server" and you want your client to tell your server to go fetch it 
directly then the "stream.url" param is what you need -- that is unrelated 
to wether you use StreamingUpdateSolrServer or not.

Thinking about it some more, i suspect the reason you might be seeing a 
delay when using StreamingUpdateSolrServer is because of this bug...

   https://issues.apache.org/jira/browse/SOLR-1990

...if there are no actual documents in your UpdateRequest (because you are 
using the stream.url param) then the StreamingUpdateSolrServer blocks 
until all other requests are done, then delegates to the super class (so 
it never actaully puts your indexing requests in a buffered queue, it just 
delays and then does them immediately)

Not sure of a good way arround this off the top of my head, but i'll note 
it in SOLR-1990 as another problematic use case that needs dealt with.


-Hoss


Mime
View raw message