lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shawn Heisey <apa...@elyograg.org>
Subject Re: Getting error while excuting full import
Date Tue, 18 Apr 2017 13:38:46 GMT
On 4/18/2017 12:58 AM, ankur.168 wrote:
> Hi Erick,
>
> Thanks for replying, As you suggest I can use solrJ to map RDBMS fetched
> data and index/search it later on. but DIH gives multi db connection for
> full import and other benefits.
> Does solrJ supports this or we need to put efforts to make a multithreaded
> connection pool similar to DIH?

Each DIH handler is single-threaded.  I have no idea what you are
talking about when you mention multi-threaded in conjunction with DIH. 
You can have multiple handlers, and execute all of them in parallel, but
each one can only execute one import at a time, and that import will
only run with one thread.  Within the limitations caused by running
single-threaded, DIH is quite efficient.

With SolrJ, you can do anything the Java language permits you to do,
including very efficient handling of multiple threads, using multiple
databases, and pretty much anything else you can dream up ... but you
must write all the code.

Safety and good performance in a multi-threaded Java program is an art
form.  I hesitate to say that it's *difficult*, but it does create
challenges that may not be trivial to overcome.

The various SolrClient objects from SolrJ are thread-safe, although you
typically must create a custom HttpClient object to use for SolrClient
creation if you're going to be using a lot of threads, because
HttpClient defaults are to only allow a very minimal number of threads.

I've got no experience at all with parent/child documents, so this part
of your troubles is a mystery to me.

Thanks,
Shawn


Mime
View raw message