calcite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@gmail.com>
Subject Re: About Stream SQL
Date Sat, 06 Feb 2016 09:28:41 GMT
Excellent! I missed the punctuations in the todo list.

What kind of strategies do you have in mind to handle events that arrive
too late? I see
1. dropping of late events
2. computing an updated window result for each late arriving
element (implies that the window state is stored for a certain period
before it is discarded)
3. computing a delta to the previous window result for each late arriving
element (requires window state as well, not applicable to all aggregation
types)

It would be nice if strategies to handle late-arrivers could be defined in
the query.

I think the plans of the Flink community are quite well aligned with your
ideas for SQL on Streams.
Should we start by updating / extending the Stream document on the Calcite
website to include the new window definitions (TUMBLE, HOP) and a
discussion of punctuations/watermarks/time bounds?

Fabian






2016-02-06 2:35 GMT+01:00 Julian Hyde <jhyde@apache.org>:

> Let me rephrase: The *majority* of the literature, of which I cited
> just one example, calls them punctuation, and a couple of recent
> papers out of Mountain View doesn't change that.
>
> There are some fine distinctions between punctuation, heartbeats,
> watermarks and rowtime bounds, mostly in terms of how they are
> generated and propagated, that matter little when planning the query.
>
> On Fri, Feb 5, 2016 at 5:18 PM, Ted Dunning <ted.dunning@gmail.com> wrote:
> > On Fri, Feb 5, 2016 at 5:10 PM, Julian Hyde <jhyde@apache.org> wrote:
> >
> >> Yes, watermarks, absolutely. The "to do" list has "punctuation", which
> >> is the same thing. (Actually, I prefer to call it "rowtime bound"
> >> because it is feels more like a dynamic constraint than a piece of
> >> data, but the literature[1] calls them punctuation.)
> >>
> >
> > Some of the literature calls them punctuation, other literature [1] calls
> > them watermarks.
> >
> > [1] http://www.vldb.org/pvldb/vol8/p1792-Akidau.pdf
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message