spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reynold Xin <r...@databricks.com>
Subject Re: SPARK-SQL: Pattern Detection on Live Event or Archived Event Data
Date Thu, 03 Mar 2016 02:42:41 GMT
SQL is very common and even some business analysts learn them. Scala and
Python are great, but the easiest language to use is often the languages a
user already knows. And for a lot of users, that is SQL.

On Wednesday, March 2, 2016, Jerry Lam <chilinglam@gmail.com> wrote:

> Hi guys,
>
> FYI... this wiki page (StreamSQL: https://en.wikipedia.org/wiki/StreamSQL)
> has some histories related Event Stream Processing and SQL.
>
> Hi Steve,
>
> It is difficult to ask your customers that they should learn a new
> language when they are not programmers :)
> I don't know where/why they learn SQL-like languages. Do business schools
> teach SQL??
>
> Best Regards,
>
> Jerry
>
>
> On Wed, Mar 2, 2016 at 10:03 AM, Steve Loughran <stevel@hortonworks.com
> <javascript:_e(%7B%7D,'cvml','stevel@hortonworks.com');>> wrote:
>
>>
>> > On 1 Mar 2016, at 22:25, Jerry Lam <chilinglam@gmail.com
>> <javascript:_e(%7B%7D,'cvml','chilinglam@gmail.com');>> wrote:
>> >
>> > Hi Reynold,
>> >
>> > You are right. It is about the audience. For instance, in many of my
>> cases, the SQL style is very attractive if not mandatory for people with
>> minimum programming knowledge.
>>
>> but SQL skills instead. Which is just relational set theory with a
>> syntax, Structured English Query Language from the IBM R project of the mid
>> 1970s (\cite{Gray et al, An evaluation of System R})
>>
>> If you look at why SQL snuck back in as a layer atop the "Post-SQL
>> systems", it's
>>
>> (a) tooling
>> (b) declarative queries can be optimised by query planners
>> (c) a lot of people who do queries on existing systems can migrate to the
>> new platforms. This is why FB wrote Hive; their PHP GUI teams didn't want
>> to learn Java.
>>
>>
>> > SQL has its place for communication. Last time I show someone spark
>> dataframe-style, they immediately said it is too difficult to use. When I
>> change it to SQL, they are suddenly happy and say how you do this. It
>> sounds stupid but that's what it is for now.
>> >
>>
>> try showing the python syntax. Python is an easier language to learn, and
>> its list comprehensions are suspiciously close to applied set theory.
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>> <javascript:_e(%7B%7D,'cvml','dev-unsubscribe@spark.apache.org');>
>> For additional commands, e-mail: dev-help@spark.apache.org
>> <javascript:_e(%7B%7D,'cvml','dev-help@spark.apache.org');>
>>
>>
>

Mime
View raw message