lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jai <jai4l...@gmail.com>
Subject slow reindexing
Date Thu, 03 Oct 2013 07:00:29 GMT
what i am doing?
i am querying from one core and reindexing data to another core.

Why?
i am querying using regular expression, it give me results but do not tell
how many unique values found and with their individual counts. (facet). i
am querying and reindexing in another core on same machine with field to
store only matched string.

Schema:
Schema is like this


<field name="Type" type="string" indexed="true" stored="true" multiValued=
"false"/> <field name="Source" type="string" indexed="true" stored="true"
multiValued="false" /> <field name="Category" type="string" indexed="true"
stored="true" multiValued="false" /> <field name="LineData" type="text"
indexed="true" stored="true" multiValued="false"/>


4 more similar text fields. these text fields are lines from text files and
are not big.

i am querying and reindexing using 20 threads. config file has default max
threads (50) for update. increasing my thread count or this default value
is not helping.
Also increased auto commit time to 3 hours. and tried increasing rambuffer
size, nothing has helped.

its taking 11 minutes to index 1 Million documents from one core to another
core.

this is far less than the benchmark and performance data on wiki.

can you please guide me where i can improve indexing speed.

setup: solr 4.2
intel xenon machine (quard core) 8GB RAM and SSD harddisk.

thanks and regards
jai

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message