spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jackey Lee (JIRA)" <>
Subject [jira] [Commented] (SPARK-24630) SPIP: Support SQLStreaming in Spark
Date Thu, 02 Aug 2018 13:25:00 GMT


Jackey Lee commented on SPARK-24630:

SQLStreaming is actually based on StructStreaming, thus, most of the syntax
that are supported by StructStreaming can also be supported in
SQLStreaming. Similarly, those queries, that are not supported in
StructStreaming, are also not supported in SQLStreaming.
Which means these queries are not supported in SQLStreaming:

> SPIP: Support SQLStreaming in Spark
> -----------------------------------
>                 Key: SPARK-24630
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: Structured Streaming
>    Affects Versions: 2.2.0, 2.2.1
>            Reporter: Jackey Lee
>            Priority: Minor
>              Labels: SQLStreaming
>         Attachments: SQLStreaming SPIP.pdf
> At present, KafkaSQL, Flink SQL(which is actually based on Calcite), SQLStream, StormSQL
all provide a stream type SQL interface, with which users with little knowledge about streaming, 
can easily develop a flow system processing model. In Spark, we can also support SQL API based
on StructStreamig.
> To support for SQL Streaming, there are two key points: 
> 1, Analysis should be able to parse streaming type SQL. 
> 2, Analyzer should be able to map metadata information to the corresponding 
> Relation. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message