lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Varun Thacker <varunthacker1...@gmail.com>
Subject Re: (Issue) How improve solr facet performance
Date Fri, 23 May 2014 10:52:33 GMT
@Amit - https://cwiki.apache.org/confluence/display/solr/DocValues


On Fri, May 23, 2014 at 4:15 PM, Jack Krupansky <jack@basetechnology.com>wrote:

>   (In the future, please submit this type of request to the solr-user
> email list. Thanks.)
>
> How much system memory is available for OS file system caching? If your
> index (30G) doesn't fit in memory, that can cause thrashing.
>
> I would suggest that 320 million docs is way too much for a single Solr
> node – performance degradation of faceting is one example of the effects of
> doing so. 100 million would be a more reasonable upper bound.
>
> -- Jack Krupansky
>
>  *From:* Alice.H.Yang (mis.cnsh04.Newegg) 41493 <Alice.H.Yang@newegg.com>
> *Sent:* Friday, May 23, 2014 5:45 AM
> *To:* mailto:dev@lucene.apache.org <dev@lucene.apache.org>
> *Subject:* (Issue) How improve solr facet performance
>
>
> Hi, Solr Developers
>
>
>
>        We are blocked by solr facet performance when query hits many
> documents. (about 10,000,000)
>
>
>
> Details as follow:
>
>        I have single solr node (solr version is 4.6 ), total has
> 320,000,000 documents( local index is about 30G)
>
>
>
> the index directory as follow:
>
>
>  The cache configure as this:
>
> <filterCache class="solr.FastLRUCache"
>
>                  size="1024000"
>
>                 initialSize="1024000"
>
>                  autowarmCount="102400"/>
>
>
>
> <queryResultCache class="solr.LRUCache"
>
>                      size="1024000"
>
>                      initialSize="1024000"
>
>                      autowarmCount="102400"/>
>
>
>
> <documentCache class="solr.LRUCache"
>
>                   size="30720"
>
>                    initialSize="30720"
>
>                    autowarmCount="3072"/>
>
>
>
>
>
> When we query the field eg. default_search: type or default_search: retail,
>
>
> in the response , numfound is about one million and hits cache, QTime is “
> <int name="QTime">2</int>” OR “<int name="QTime">1</int>.”
>
>
>
> But when we add several facet.field to do facet , QTime  increase to
> 220ms or more.
>
>
>
> We add the parameter facet.threads=20, but the improvement is not obvious,
> and unstable.
>
>
>
> Do you have some advice on how improve the facet performance when hit many
> documents.
>
>
>
>
>
>
>
> Best Regards,
>
> *Alice Yang*
>
> *+86-021-51530666 <%2B86-021-51530666>*41493*
>
> *Floor 19,KaiKai Plaza,888,Wanhandu Rd,Shanghai(200042)*
>
>
>
> *ONCE YOU KNOW, YOU NEWEGG.*
>
>
>
> *CONFIDENTIALITY NOTICE: This email and any files transmitted with it may
> contain privileged or otherwise confidential information. It is intended
> only for the person or persons to whom it is addressed. If you received
> this message in error, you are not authorized to read, print, retain, copy,
> disclose, disseminate, distribute, or use this message any part thereof or
> any information contained therein. Please notify the sender immediately and
> delete all copies of this message. Thank you in advance for your
> cooperation.*
>
>
> 保密注意:此邮件及其附随文件可能包含了保密信息。该邮件的目的是发送给指定收件人。如果您非指定收件人而错误地收到了本邮件,您将无权阅读、打印、保存、复制、泄露、传播、分发或使用此邮件全部或部分内容或者邮件中包含的任何信息。请立即通知发件人,并删除该邮件。感谢您的配合!
>
>
>



-- 


Regards,
Varun Thacker
http://www.vthacker.in/

Mime
View raw message