From solr-user-return-74787-apmail-lucene-solr-user-archive=lucene.apache.org@lucene.apache.org Fri Nov 16 14:51:49 2012 Return-Path: X-Original-To: apmail-lucene-solr-user-archive@minotaur.apache.org Delivered-To: apmail-lucene-solr-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2F0BBD2E8 for ; Fri, 16 Nov 2012 14:51:49 +0000 (UTC) Received: (qmail 40848 invoked by uid 500); 16 Nov 2012 14:51:45 -0000 Delivered-To: apmail-lucene-solr-user-archive@lucene.apache.org Received: (qmail 40779 invoked by uid 500); 16 Nov 2012 14:51:45 -0000 Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: solr-user@lucene.apache.org Delivered-To: mailing list solr-user@lucene.apache.org Received: (qmail 40749 invoked by uid 99); 16 Nov 2012 14:51:45 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 16 Nov 2012 14:51:45 +0000 X-ASF-Spam-Status: No, hits=0.0 required=5.0 tests=MIME_QP_LONG_LINE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [178.21.113.82] (HELO mail.openindex.io) (178.21.113.82) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 16 Nov 2012 14:51:37 +0000 Received: from localhost (localhost [127.0.0.1]) by mail.openindex.io (Postfix) with ESMTP id 0CF78FC002 for ; Fri, 16 Nov 2012 14:56:31 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at mail.openindex.io Received: from mail.openindex.io ([127.0.0.1]) by localhost (mail.openindex.io [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id PCoC1oA5PuRO for ; Fri, 16 Nov 2012 14:56:30 +0000 (UTC) Received: from mail.openindex.io (localhost [127.0.0.1]) by mail.openindex.io (Postfix) with ESMTP id 6766DFC001 for ; Fri, 16 Nov 2012 14:56:30 +0000 (UTC) Subject: Reduce QueryComponent prepare time From: =?utf-8?Q?Markus_Jelsma?= To: =?utf-8?Q?solr-user=40lucene=2Eapache=2Eorg?= Date: Fri, 16 Nov 2012 14:56:30 +0000 Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Priority: 3 (Normal) X-Mailer: Zarafa 7.0.7-34256 Message-Id: X-Virus-Checked: Checked by ClamAV on apache.org Hi, We're seeing high prepare times for the QueryComponent, obviously due to the vast amount of field and queries. It's common to have a prepare time of 70-80ms while the process times drop significantly due to warmed searchers, OS cache etc. The prepare time is a recurring issue and i'd hope if there are people here that can share some thoughts or hints. We're using a recent check out on a 10 node test cluster with SSD's (although this is no IO issue) and edismax on about a hundred different fields, this includes phrase searches over most of those fields and SpanFirst queries on about 25 fields. We'd like to see how we can avoid doing the same prepare procedure over and over again ;) Thanks, Markus