From drill-dev-return-39-apmail-incubator-drill-dev-archive=incubator.apache.org@incubator.apache.org Sun Aug 26 23:04:02 2012 Return-Path: X-Original-To: apmail-incubator-drill-dev-archive@minotaur.apache.org Delivered-To: apmail-incubator-drill-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5382DDCAF for ; Sun, 26 Aug 2012 23:04:02 +0000 (UTC) Received: (qmail 27631 invoked by uid 500); 26 Aug 2012 23:04:02 -0000 Delivered-To: apmail-incubator-drill-dev-archive@incubator.apache.org Received: (qmail 27601 invoked by uid 500); 26 Aug 2012 23:04:02 -0000 Mailing-List: contact drill-dev-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: drill-dev@incubator.apache.org Delivered-To: mailing list drill-dev@incubator.apache.org Received: (qmail 27592 invoked by uid 99); 26 Aug 2012 23:04:02 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 26 Aug 2012 23:04:02 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of tnachen@gmail.com designates 209.85.210.47 as permitted sender) Received: from [209.85.210.47] (HELO mail-pz0-f47.google.com) (209.85.210.47) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 26 Aug 2012 23:03:52 +0000 Received: by daks35 with SMTP id s35so2278835dak.6 for ; Sun, 26 Aug 2012 16:03:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:references:from:content-type:x-mailer:in-reply-to :message-id:date:to:content-transfer-encoding:mime-version; bh=A0g+biHvWao0f+mkWBFLg+1RTeTYg3uwb6SC8TDsNBY=; b=ETaPyWnHHWRdnH5+k2o3V/ZAFpOmRREoB0DM1tH4DbuOAyfw4HOGMIvk0wckMWaHKI ZCNtVlrtKWPa8+aMvgPmkSmL5PXjPLFp7yC1eLvriEFnetQupEfneknbuUewIpDjH0/C EMJ93Pj3hMFjslmC2jvZv6UJWZ0KPe1C5Ium3gxXTqfdixTnnR95OfBO/SYcVCTKZugf vIp6ygygi9dUJXLQnmtCFm8+DqM8TNyyv128OBKKJLW42LFzRfUp4AU/JxbaULDoaOcp 6cQQdcyhuZletgdIsRkK/n2kRSOCMZYQGU1GAUYEJqdQ04OsA3bE75a32AbbS4JIg/pp cjXw== Received: by 10.68.203.67 with SMTP id ko3mr29999871pbc.126.1346022211539; Sun, 26 Aug 2012 16:03:31 -0700 (PDT) Received: from [10.78.123.220] (mobile-166-147-093-097.mycingular.net. [166.147.93.97]) by mx.google.com with ESMTPS id pf10sm13280542pbc.56.2012.08.26.16.03.26 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 26 Aug 2012 16:03:29 -0700 (PDT) Subject: Re: DrQL grammar and parser References: From: Timothy Chen Content-Type: text/plain; charset=us-ascii X-Mailer: iPhone Mail (9B206) In-Reply-To: Message-Id: Date: Sun, 26 Aug 2012 16:02:29 -0700 To: "drill-dev@incubator.apache.org" Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (1.0) Sent from my iPhone On Aug 26, 2012, at 3:21 PM, Camuel Gilyadov wrote: > On Mon, Aug 27, 2012 at 12:22 AM, Ted Dunning wrot= e: >=20 >> Camuel, >>=20 >> Do you have a grammar test suite that demonstrates the range of >> expressions? >>=20 >=20 > I don't have an exhaustive one. But with OpenDremel I tested it with all > sample queries given in the BQL manual and then adding my own for various > cases. I can do similar thing here as well as develop grammar unit tests > (with ANTLR's gunit). >=20 >=20 >>=20 >> Also, I believe that some have a goal to use additional languages besides= >> SQL like languages. A limited version of pig, for instance, would be ver= y >> interesting. To do this, it will be important to have a logical plan >> structure that is common for different syntaxes and is not limited to the= >> idiosyncracies of any particular syntax. >>=20 >> How do you think that should be handled? Do you have an idea for a logic= al >> plan structure? >>=20 >=20 > Yes I have, but I afraid it is a radical idea. I think queries must be > directly compiled into generic C source code. That way in worst case, a > completely separate compiler could be developed for some exotic query > language and backend could be still reused. On other extreme, dialects > could be handled in the grammar itself yielding same AST so frontend code > should not even be aware.Between these two extremes two different grammars= > should be used, both yielding object models which could then handled by > same code generator (so code generator is reused). Not to say that this > approach simplifies the backend and it is most tricky part to implement as= > it must be scaled and secured and outlives single query or query session. >=20 > I think an attempt to design a logical query language expressive enough to= > cover many existing and future query languages would not be all that easy > going... >=20 >=20 >>=20 >> On Sun, Aug 26, 2012 at 4:11 PM, Camuel Gilyadov wrote= : >>=20 >>> I've written and attached ANTLR grammar for DrQL which I assume is same >> as >>> BigQuery language described in Query Reference on BigQuery website. This= >>> grammar includes AST production rules. >>>=20 >>>=20 >>>=20 >>=20