lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From SolrLover <bbar...@gmail.com>
Subject Re: Streaming Updates Using HttpSolrServer.add(Iterator) In Solr 4.3
Date Mon, 29 Jul 2013 23:51:23 GMT
I am indexing more than 300 million records, it takes less than 7 hours to
index all the records..

Send the documents in batches and also use CUSS (ConcurrentUpdateSolrServer)
for multi threading support.

Ex: 

 ConcurrentUpdateSolrServer server= new
ConcurrentUpdateSolrServer(solrServer, queueSize,
					threadCount);
		List<SolrInputDocument> solrDocList = new ArrayList<SolrInputDocument>();
     While (loop) {
	solrDocList.add(doc); --> Add the documents to array
       if(count >=100){
           server.add(solrDocList); --> Add documents to SOLR in batches
       }
     count++;
    }
server.commit(); --> Commit after adding all the documents 




--
View this message in context: http://lucene.472066.n3.nabble.com/Streaming-Updates-Using-HttpSolrServer-add-Iterator-In-Solr-4-3-tp4081171p4081172.html
Sent from the Solr - User mailing list archive at Nabble.com.

Mime
View raw message