From solr-user-return-119474-apmail-lucene-solr-user-archive=lucene.apache.org@lucene.apache.org Tue Jan 5 00:42:49 2016 Return-Path: X-Original-To: apmail-lucene-solr-user-archive@minotaur.apache.org Delivered-To: apmail-lucene-solr-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3A6FA1820E for ; Tue, 5 Jan 2016 00:42:49 +0000 (UTC) Received: (qmail 37008 invoked by uid 500); 5 Jan 2016 00:42:45 -0000 Delivered-To: apmail-lucene-solr-user-archive@lucene.apache.org Received: (qmail 36940 invoked by uid 500); 5 Jan 2016 00:42:44 -0000 Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: solr-user@lucene.apache.org Delivered-To: mailing list solr-user@lucene.apache.org Received: (qmail 36928 invoked by uid 99); 5 Jan 2016 00:42:44 -0000 Received: from Unknown (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 05 Jan 2016 00:42:44 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id F415E1A04CC for ; Tue, 5 Jan 2016 00:42:43 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 5.013 X-Spam-Level: ***** X-Spam-Status: No, score=5.013 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FUZZY_MILLION=2.599, KAM_COUK=1.1, URIBL_BLOCKED=0.001, URI_HEX=1.313] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (1024-bit key) header.d=messagingengine.com Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id kScKWpKx1TvZ for ; Tue, 5 Jan 2016 00:42:32 +0000 (UTC) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTPS id 33D5E2304E for ; Tue, 5 Jan 2016 00:42:28 +0000 (UTC) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 62DB620BE4 for ; Mon, 4 Jan 2016 19:42:25 -0500 (EST) Received: from web6 ([10.202.2.216]) by compute1.internal (MEProxy); Mon, 04 Jan 2016 19:42:25 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-sasl-enc:x-sasl-enc; s=smtpout; bh=OPLNGyzFiSVpsP2 ceWn3KeqRI9Q=; b=o0XlHbWWyeUWVARvvgj6GXbwht7cxZyqEbRiUN/DuZ/xVAy CTG/VfUOUDmhTXMEziVXiYuRCvs1Xa4+mAxSaCXJq1BR+NFR2ywPPBVjaW+WJ158 GqOwv1e9Yf6nChg0YyGMl3yMHrHLWIzrK6pnfATgs6Xz4h0ZXXX6S6YI8xvc= Received: by web6.nyi.internal (Postfix, from userid 99) id 2C97846347; Mon, 4 Jan 2016 19:42:25 -0500 (EST) Message-Id: <1451954545.1621804.482920698.2232CF7A@webmail.messagingengine.com> X-Sasl-Enc: sq1jUC74yW6EUFMGDli1O1Tv3AfHzcKcUmgGJrv0+MQT 1451954545 From: Upayavira To: solr-user@lucene.apache.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Mailer: MessagingEngine.com Webmail Interface - ajax-42cc5299 In-Reply-To: References: <1451904093127-4248360.post@n3.nabble.com> <1451904566.1436758.482184242.1A2E49BC@webmail.messagingengine.com> <1451912624530-4248388.post@n3.nabble.com> Subject: Re: how to search miilions of record in solr query Date: Tue, 05 Jan 2016 00:42:25 +0000 Yes, because only a small portion of that 250ms is spent in the query parser. Most of it, i would suggest, is spent retrieving and merging posting lists. In an inverted index (which Lucene is), you store the list of documents matching a term against that term - that is your postings list. When you search against multiple terms, Lucene needs to merge those into a definitive list of matching documents, and for large numbers of terms, that can be costly. Upayavira On Mon, Jan 4, 2016, at 04:29 PM, Erick Erickson wrote: > Best of luck with that ;). 250ms isn't bad at all for "searching > millions of IDs". > Frankly, I'm not at all sure where I'd even start. With millions of > search > terms, I'd have to profile the application to see where it was spending > the > time before even starting. > > Best, > Erick > > On Mon, Jan 4, 2016 at 5:03 AM, Mugeesh Husain wrote: > >>>This is not a use-case to which Lucene lends itself. However, if you > >>>must, I would try the terms query parser, which I believe is used like > >>>this: {!terms f=id}2,3,6,7 > > > > I did try terms query parser like above, but the problem is performance, i > > am getting result 250ms but i am looking for a parser which give result > > within 50ms. > > > > I am also looking for custom query parser but i dont know which way i should > > used that. > > > > > > > > -- > > View this message in context: http://lucene.472066.n3.nabble.com/how-to-search-miilions-of-record-in-solr-query-tp4248360p4248388.html > > Sent from the Solr - User mailing list archive at Nabble.com.