calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Pilloud <>
Subject Re: Google BigQuery - zetasql parser/analyzer
Date Thu, 02 May 2019 21:03:46 GMT
Sorry for the delay. ZetaSQL is not a threat to Calcite. This project
is Google's internal efforts to standardize around a single SQL
dialect. It is a parser and analyzer, Calcite is much more. We also
have an implementation test suite, consisting of a large set of sample
queries and expected results, on our roadmap for open sourcing.

What we would like to contribute now is a thin API by which Calcite
can talk to ZetaSQL for parsing queries. We have a prototype of this
that we plan to contribute to Beam, but it seems like it would be
generally useful to other Calcite users. (The interface would likely
be useful for wrapping other parsers as well.) This will rely on the
ZetaSQL C++ parser, which creates significant limitations around
portability but gives the ability to strictly match BigQuery's
"Standard SQL" dialect.

Once we open source the test suite, I expect we would be interested in
using that to improve Babel's BigQuery mode. We do not have plans to
write a pure Java ZetaSQL parser, but have a need for one. ZetaSQL is
constantly evolving and has a significant number of teams working on
it inside Google. I don't think Babel can replace it or that the
projects can be merged, but we see the long term value in cooperating
with you to improve the Babel parser.


From: Julian Hyde <>
Date: Fri, Apr 26, 2019 at 11:08 AM
To: dev

> Thanks for chiming in, Andrew.
> I’m a little confused about the purpose of ZetaSQL currently. (I’m sure it will start
to click as time goes on.)
> My initial reaction is that ZetaSQL has similar goals to Calcite - to decompose the traditional
DBMS into components from which people can roll their own database, adding their own planning,
engine, data format, algorithms, etc. That is a revolutionary vision which I fully support.
> Calcite should not feel threatened by ZetaSQL. As Google pursues this vision of the deconstructed
database, there will be more opportunities for us to work with the newly exposed components.
Our integration with Beam is an example.
> I second Michael’s question “could you clarify what you’re hoping to contribute
to Babel?” Would you contribute a (thin) API by which Babel could talk to ZetaSQL’s parser,
or code that parses various dialects in native Java.
> It’s probably too much to hope for Babel to evolve into the java port of ZetaSQL, but
I would be open to the idea. And if that doesn’t happen, another way of cooperating that
would be fruitful would be for the projects to have separate code but share a test suite.
> Julian
> > On Apr 26, 2019, at 6:14 AM, Michael Mior <> wrote:
> >
> > This is basically the point of the Babel parser, to be as liberal as
> > possible in what SQL is accepted and configurable where things
> > necessarily conflict between different implementations.
> >
> > Andrew, could you clarify what you're hoping to contribute to Babel? I
> > think support for any dialect of SQL will generally be welcomed :)
> >
> > --
> > Michael Mior
> >
> >
> > Le jeu. 25 avr. 2019 à 23:39, Lai Zhou <> a écrit :
> >>
> >> Good news.
> >> I found Julian created a new issue about various sql parsing.
> >>
> >> I think it may be helpful to support various sql parsing in a unified
> >> manner.
> >> Calcite did use the SqlConformance to enable various SQL compatibility
> >> modes,
> >> but it's not enough to solve all the problems.
> >> Except sql parsing, different sql engines also have differences on
> >> type inferring,
> >> type checking and  implicit casting.
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> Andrew Pilloud <> 于2019年4月26日周五
> >>
> >>> I was intending to send an email about this, thanks for starting the
> >>> discussion. I'm on the team at Google that is open sourcing ZetaSQL.
> >>> It is a C++ SQL parser used internally for the BigQuery standard sql
> >>> parser, among other things.
> >>>
> >>> We've open source the Java frontend and Rui currently working on a
> >>> adapter between ZetaSQL and Calcite for Apache Beam, but we'd probably
> >>> be interested in contributing it directly to Calcite Bable at some
> >>> point. Any thoughts on that?
> >>>
> >>> Andrew
> >>>
> >>> On Thu, Apr 25, 2019 at 4:08 PM Kevin Risden <>
> >>>>
> >>>>
> >>>>
> >>>> Saw this come by on twitter not too long ago and figured share here
> >>>> it definitely overlaps with Calcite.
> >>>>
> >>>> Kevin Risden
> >>>

View raw message