On 1/22/2016 10:29 AM, Brian Narsi wrote:
> What are the various ways DataImportHandler can be scaled?
I'm not very familiar with how DIH interacts with SolrCloud. I know you
can use it with SolrCloud, but nothing else. Assuming you're not
running SolrCloud, the following information will apply:
The only way I know of to scale DIH performance on a single Solr core is
to define multiple DIH handlers, each of which imports a subset of your
data. Then you can run multiple imports simultaneously. A single DIH
handler can only run one import at a time, and it will be single-threaded.
If you've got a sharded index, each shard would be in its own core, and
therefore each one would have its own instance of your dataimport
handler with its own config, so you can run an import on each shard
simultaneously. This is what happens on my index when I need to fully
rebuild it.
Thanks,
Shawn
|