lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jack Krupansky" <j...@basetechnology.com>
Subject Re: (Issue) How improve solr facet performance
Date Fri, 23 May 2014 10:45:12 GMT
(In the future, please submit this type of request to the solr-user email list. Thanks.)

How much system memory is available for OS file system caching? If your index (30G) doesn't
fit in memory, that can cause thrashing.

I would suggest that 320 million docs is way too much for a single Solr node – performance
degradation of faceting is one example of the effects of doing so. 100 million would be a
more reasonable upper bound.

-- Jack Krupansky

From: Alice.H.Yang (mis.cnsh04.Newegg) 41493 
Sent: Friday, May 23, 2014 5:45 AM
To: mailto:dev@lucene.apache.org 
Subject: (Issue) How improve solr facet performance

Hi, Solr Developers

 

       We are blocked by solr facet performance when query hits many documents. (about 10,000,000)

 

Details as follow:

       I have single solr node (solr version is 4.6 ), total has 320,000,000 documents( local
index is about 30G) 

 

the index directory as follow:




The cache configure as this:

<filterCache class="solr.FastLRUCache"

                 size="1024000"

                initialSize="1024000"

                 autowarmCount="102400"/>

 

<queryResultCache class="solr.LRUCache"

                     size="1024000"

                     initialSize="1024000"

                     autowarmCount="102400"/>

 

<documentCache class="solr.LRUCache"

                  size="30720"

                   initialSize="30720"

                   autowarmCount="3072"/>

 

 

When we query the field eg. default_search: type or default_search: retail, 

in the response , numfound is about one million and hits cache, QTime is “ <int name="QTime">2</int>”
OR “<int name="QTime">1</int>.”

 

But when we add several facet.field to do facet , QTime  increase to 220ms or more.

 

We add the parameter facet.threads=20, but the improvement is not obvious, and unstable.

 

Do you have some advice on how improve the facet performance when hit many documents. 

 

 

 

Best Regards,

Alice Yang

+86-021-51530666*41493

Floor 19,KaiKai Plaza,888,Wanhandu Rd,Shanghai(200042)

 

ONCE YOU KNOW, YOU NEWEGG.

 

CONFIDENTIALITY NOTICE: This email and any files transmitted with it may contain privileged
or otherwise confidential information. It is intended only for the person or persons to whom
it is addressed. If you received this message in error, you are not authorized to read, print,
retain, copy, disclose, disseminate, distribute, or use this message any part thereof or any
information contained therein. Please notify the sender immediately and delete all copies
of this message. Thank you in advance for your cooperation.

保密注意:此邮件及其附随文件可能包含了保密信息。该邮件的目的是发送给指定收件人。如果您非指定收件人而错误地收到了本邮件,您将无权阅读、打印、保存、复制、泄露、传播、分发或使用此邮件全部或部分内容或者邮件中包含的任何信息。请立即通知发件人,并删除该邮件。感谢您的配合!

 

Mime
View raw message