From solr-user-return-77317-apmail-lucene-solr-user-archive=lucene.apache.org@lucene.apache.org Thu Jan 10 14:05:37 2013 Return-Path: X-Original-To: apmail-lucene-solr-user-archive@minotaur.apache.org Delivered-To: apmail-lucene-solr-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CF032D75F for ; Thu, 10 Jan 2013 14:05:37 +0000 (UTC) Received: (qmail 73068 invoked by uid 500); 10 Jan 2013 14:05:34 -0000 Delivered-To: apmail-lucene-solr-user-archive@lucene.apache.org Received: (qmail 73012 invoked by uid 500); 10 Jan 2013 14:05:34 -0000 Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: solr-user@lucene.apache.org Delivered-To: mailing list solr-user@lucene.apache.org Received: (qmail 73002 invoked by uid 99); 10 Jan 2013 14:05:34 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Jan 2013 14:05:34 +0000 X-ASF-Spam-Status: No, hits=-5.0 required=5.0 tests=RCVD_IN_DNSWL_HI,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of shahard@checkpoint.com designates 194.29.34.68 as permitted sender) Received: from [194.29.34.68] (HELO smtp.checkpoint.com) (194.29.34.68) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Jan 2013 14:05:23 +0000 Received: from il-ex01.ad.checkpoint.com (il-ex01.ad.checkpoint.com [194.29.34.26]) by smtp.checkpoint.com (8.13.8/8.13.8) with ESMTP id r0AE4vXB021357; Thu, 10 Jan 2013 16:04:57 +0200 X-CheckPoint: {50EEC85C-0-1B221DC2-1FFFF} Received: from DAG-EX10.ad.checkpoint.com (194.29.34.150) by il-ex01.ad.checkpoint.com (194.29.34.26) with Microsoft SMTP Server (TLS) id 8.3.297.1; Thu, 10 Jan 2013 16:04:57 +0200 Received: from IL-EX10.ad.checkpoint.com ([169.254.2.18]) by DAG-EX10.ad.checkpoint.com ([fe80::80df:1c2c:3d29:3748%11]) with mapi id 14.02.0328.009; Thu, 10 Jan 2013 16:04:57 +0200 From: Shahar Davidson To: "solr-user@lucene.apache.org" CC: "steff@designware.dk" Subject: RE: CoreAdmin STATUS performance Thread-Topic: CoreAdmin STATUS performance Thread-Index: Ac3ufyLCnfX13J7iRVOjUT0wHSyZNgAPMuqAABUKZ/AAAP2wgAAGYqdwAANvDYA= Date: Thu, 10 Jan 2013 14:04:56 +0000 Message-ID: <7E2AC591E7B3B146BB21F295279AC26C070FF49E@IL-EX10.ad.checkpoint.com> References: <7E2AC591E7B3B146BB21F295279AC26C070FF226@IL-EX10.ad.checkpoint.com> <50EE10AF.2080105@elyograg.org> <7E2AC591E7B3B146BB21F295279AC26C070FF379@IL-EX10.ad.checkpoint.com> <50EEA489.2080208@designware.dk> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [91.90.128.18] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-KSE-AntiSpam-Interceptor-Info: protection disabled X-KSE-Antivirus-Interceptor-Info: scan successful X-KSE-Antivirus-Info: Clean X-Virus-Checked: Checked by ClamAV on apache.org Hi Per, =20 Thanks for your reply! That's a very interesting approach. In your system, how are the collections created? In other words, are the co= llections created dynamically upon an update (for example, per new day)? If they are created dynamically, who handles their creation (client/server)= and how is it done? I'd love to hear more about it! Appreciate your help, Shahar. -----Original Message----- From: Per Steffensen [mailto:steff@designware.dk] Sent: Thursday, January 10, 2013 1:23 PM To: solr-user@lucene.apache.org Subject: Re: CoreAdmin STATUS performance On 1/10/13 10:09 AM, Shahar Davidson wrote: > search request, the system must be aware of all available cores in=20 > order to execute distributed search on_all_ relevant cores For this purpose I would definitely recommend that you go "SolrCloud". Further more we do something "ekstra": We have several collections each containing data from a specific period in = time - timestamp of ingoing data decides which collection it is indexed int= o. One important search-criteria for our clients are search on timestamp-in= terval. Therefore most searches can be restricted to only consider a subset= of all our collections. Instead of having the logic calculating the subset= of collections to search (given the timestamp search-interval) in clients, we just let clients do "dumb" searches by givi= ng the timestamp-interval. The subset of collections to search are calculat= ed on server-side from the timestamp-interval in the search-query. We handl= e this in a Solr SearchComponent which we place "early" in the chain of Sea= rchComponents. Maybe you can get some inspiration by this approach, if it i= s also relevant for you. Regards, Per Steffensen Email secured by Check Point