calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Julian Hyde <jh...@apache.org>
Subject Re: Towards a spec for robust streaming SQL, Part 2
Date Tue, 01 Aug 2017 19:24:41 GMT
I have problems with a couple of the axioms: that a SQL object is
either a table or a stream, but not both; and that a query is bounded
if and only if it contains no unbounded streams.

I don't have problems with other axioms, such that a query is either
bounded or unbounded. And I haven't looked in detail at triggering
semantics; I don't think there will be major issues, but let's clear
up the 2 problems above first.

I have added a section "Julian’s thoughts on the fundamentals" to the
end of the document.

Julian


On Tue, Aug 1, 2017 at 6:40 AM, Fabian Hueske <fhueske@gmail.com> wrote:
> As promised, I went of the document and made some comments.
> I also added a bit of information about the current SQL support in Flink
> and its internals.
>
> Thanks, Fabian
>
> 2017-07-30 13:22 GMT+02:00 Shaoxuan Wang <wshaoxuan@gmail.com>:
>
>> Hi Tyler,
>> Thanks for putting all the efforts into a doc. It is really well written
>> and organized.
>> I like the most part. The major concern I have is about the "explicit
>> trigger". I left a few comments towards this and would like to know what
>> the others think about it.
>>
>> Regards,
>> Shaoxuan
>>
>> On Sun, Jul 30, 2017 at 4:43 PM, Fabian Hueske <fhueske@gmail.com> wrote:
>>
>> > Thanks for the great write up!
>> >
>> > I think this s very good starting point for a detailed discussion about
>> > features, syntax and semantics of streaming SQL.
>> > I'll comment on the document in the next days and describe Flink's
>> current
>> > status, our approaches (or planned approaches) and ask a couple of
>> > questions.
>> >
>> > Thanks, Fabian
>> >
>> > 2017-07-28 3:05 GMT+02:00 Julian Hyde <jhyde@apache.org>:
>> >
>> > > Tyler,
>> > >
>> > > Thanks for this. I am reading the document thoroughly and will give my
>> > > feedback in a day or two.
>> > >
>> > > Julian
>> > >
>> > > > On Jul 25, 2017, at 12:54 PM, Pramod Immaneni <
>> pramod@datatorrent.com>
>> > > wrote:
>> > > >
>> > > > Thanks for the invitation Tyler. I am sure folks who worked on the
>> > > calcite
>> > > > integration and others would be interested.
>> > > >
>> > > > On Tue, Jul 25, 2017 at 12:12 PM, Tyler Akidau
>> > > <takidau@google.com.invalid>
>> > > > wrote:
>> > > >
>> > > >> +dev@apex.apache.org, since I'm told Apex has a Calcite integration
>> > as
>> > > >> well. If anyone on the Apex side wants to join in on the fun,
your
>> > input
>> > > >> would be welcomed!
>> > > >>
>> > > >> -Tyler
>> > > >>
>> > > >>
>> > > >> On Mon, Jul 24, 2017 at 4:34 PM Tyler Akidau <takidau@apache.org>
>> > > wrote:
>> > > >>
>> > > >>> Hello Flink, Calcite, and Beam dev lists!
>> > > >>>
>> > > >>> Linked below is the second document I promised way back in
April
>> > > >> regarding
>> > > >>> a collaborative spec for streaming SQL in Beam/Calcite/Flink
(&
>> > > apologies
>> > > >>> for the delay; I thought I was nearly done a while back and
then
>> > > temporal
>> > > >>> joins expanded to something much larger than expected).
>> > > >>>
>> > > >>> To repeat what it says in the doc, my hope is that it can
serve
>> > various
>> > > >>> purposes over it's lifetime:
>> > > >>>
>> > > >>>   -
>> > > >>>   - A discussion ground for ironing out any remaining features
>> > > necessary
>> > > >>>   for supporting robust streaming semantics in Calcite SQL.
>> > > >>>
>> > > >>>   - A rough, high-level source of truth for tracking efforts
>> underway
>> > > in
>> > > >>>   support of this, currently spanning the Calcite, Flink,
and Beam
>> > > >> projects.
>> > > >>>
>> > > >>>   - A written specification of the changes that were made,
for the
>> > sake
>> > > >>>   of understanding the delta after the fact.
>> > > >>>
>> > > >>> The first and third points are, IMO, the most important. AFAIK,
>> there
>> > > are
>> > > >>> a few features missing still that need to be defined (e.g.,
>> triggers
>> > > >>> equivalents via EMIT, robust temporal join support). I'm also
>> > > proposing a
>> > > >>> clear distinction of streams and tables, which I think is
>> important,
>> > > but
>> > > >>> which I believe is not the approach most folks have been taking
in
>> > this
>> > > >>> area. Sorting out these open issues and then having a concise
>> record
>> > of
>> > > >> the
>> > > >>> solutions adopted will be important for providing a solid
streaming
>> > > >>> experience and teaching folks how to use it.
>> > > >>>
>> > > >>> At any rate, I would much appreciate it if anyone with an
interest
>> in
>> > > >> this
>> > > >>> stuff could please take a look and add comments/suggestions/
>> > references
>> > > >> to
>> > > >>> related work in flight/etc as appropriate. For now please
use
>> > > >>> comments/suggestions, but if you really want to dive in with
edit
>> > > access,
>> > > >>> let me know.
>> > > >>>
>> > > >>> The doc: http://s.apache.org/streaming-sql-spec
>> > > >>>
>> > > >>> -Tyler
>> > > >>>
>> > > >>>
>> > > >>>
>> > > >>
>> > >
>> > >
>> >
>>

Mime
View raw message