lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Bootstrapping / Full Importing using Solr Cloud
Date Wed, 09 Oct 2013 00:27:07 GMT
DIH works with SolrCloud as far as I understand. But
moving to SolrJ has several advantages:
1> you have more control over our process, beter
ability to debug etc.
2> If you can partition your data up amongst
several clients, you can probably get through your jobs
much faster.
3> You're not overloading one machine with both the
DIH bits and the indexing bits.

There are some other options, I generally prefer SolrJ
though. Others have different opinions of course.

Best,
Erick

On Tue, Oct 8, 2013 at 12:57 PM, Mark <static.void.dev@gmail.com> wrote:
> We are in the process of upgrading our Solr cluster to the latest and greatest Solr Cloud.
I have some questions regarding full indexing though. We're currently running a long job (~30
hours) using DIH to do a full index on over 10M products. This process consumes a lot of memory
and while updating can not handle any user requests.
>
> How, or what would be the best way going about this when using Solr Cloud? First off,
does DIH work with cloud? Would I need to separate out my DIH indexing machine from the machines
serving up user requests? If not going down the DIH route, what are my best options (solrj?)
>
> Thanks for the input

Mime
View raw message