spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <em...@yeikel.com>
Subject RE: Watch "Airbus makes more of the sky with Spark - Jesse Anderson & Hassene Ben Salem" on YouTube
Date Sat, 25 Apr 2020 18:39:37 GMT
Zahid, 

 

Starting with Spark 2.3.0, the Spark team introduced an experimental feature called “Continuous
Streaming”[1][2] to enter that space, but in general, Spark streaming operates using micro-batches
while Flink operates using the Continuous Flow Operator model. 

 

There are many resources online comparing the two but I am leaving you one[3] (old, but still
relevant)  so you can start looking into it.

 

Note that while I am not a subject expert, that’s the basic explanation. Until recently
we were not competing with Flink in that space, so it explains why Flink was preferred at
the time and why it would still be preferred today. We will catch up eventually.

 

[1] https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#continuous-processing


[2] https://databricks.com/blog/2018/03/20/low-latency-continuous-processing-mode-in-structured-streaming-in-apache-spark-2-3-0.html


[3] https://www.youtube.com/watch?v=Dzx-iE6RN4w&feature=emb_title

 

From: Zahid Rahman <zahidr1000@gmail.com> 
Sent: Saturday, April 25, 2020 7:36 AM
To: joerg.strebel@posteo.de
Cc: user <user@spark.apache.org>
Subject: Re: Watch "Airbus makes more of the sky with Spark - Jesse Anderson & Hassene
Ben Salem" on YouTube

 

My motive is simple . I want you (spark product  experts user)  to challenge the reason given
by Jesse Anderson for choosing flink over spark. 

 

You know the saying keep your friends close, keep your enemies even closer.

The video  only has  328 views.

It is a great educational tool to see a recent recent Use Case. Should be of compelling interest
to anyone in this field. Commercial Companies do not often share or discuss their projects
openly.

 

Incidentally Heathrow is the busiest airport in the world.

1. Because the emailing facility completed my sentence. 

 

2. I think at Heathrow the gap is less than two minutes.

 

 

On Sat, 25 Apr 2020, 09:42 Jörg Strebel, <joerg.strebel@gmail.com <mailto:joerg.strebel@gmail.com>
> wrote:

Hallo!

Well, the title of the video is actually "Airbus makes more of the sky with Flink - Jesse
Anderson & Hassene Ben Salem"and it talks about Apache Flink and specifically not about
Apache Spark.They excluded Spark Streaming for high latency reasons. 

Why are you posting this video on a Spark mailing list?

Regards

J. Strebel

Am 25.04.20 um 05:07 schrieb Zahid Rahman:

 

https://youtu.be/sYlbD_OoHhs




Backbutton.co.uk <http://Backbutton.co.uk> 

¯\_(ツ)_/¯ 
♡۶Java♡۶RMI ♡۶

Make Use Method {MUM}

makeuse.org <http://makeuse.org> 

-- 
Jörg Strebel
Aachener Straße 2
80804 München


Mime
View raw message