spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Holden Karau <hol...@pigscanfly.ca>
Subject Re: Watch "Airbus makes more of the sky with Spark - Jesse Anderson & Hassene Ben Salem" on YouTube
Date Sat, 25 Apr 2020 18:43:55 GMT
Also it’s ok if Spark and Flink evolve in different directions, were both
part of the same open source foundation. Sometimes being everything to
everyone isn’t as important as being the best at what you need.

I like to think of our relationship with other Apache projects as less
competitive and more cooperative, and focus on how can we make open source
data tools successful.

On Sat, Apr 25, 2020 at 11:40 AM <email@yeikel.com> wrote:

> Zahid,
>
>
>
> Starting with Spark 2.3.0, the Spark team introduced an experimental
> feature called “Continuous Streaming”[1][2] to enter that space, but in
> general, Spark streaming operates using micro-batches while Flink operates
> using the Continuous Flow Operator model.
>
>
>
> There are many resources online comparing the two but I am leaving you
> one[3] (old, but still relevant)  so you can start looking into it.
>
>
>
> Note that while I am not a subject expert, that’s the basic explanation.
> Until recently we were not competing with Flink in that space, so it
> explains why Flink was preferred at the time and why it would still be
> preferred today. We will catch up eventually.
>
>
>
> [1]
> https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#continuous-processing
>
> [2]
> https://databricks.com/blog/2018/03/20/low-latency-continuous-processing-mode-in-structured-streaming-in-apache-spark-2-3-0.html
>
> [3] https://www.youtube.com/watch?v=Dzx-iE6RN4w&feature=emb_title
>
>
>
> *From:* Zahid Rahman <zahidr1000@gmail.com>
> *Sent:* Saturday, April 25, 2020 7:36 AM
> *To:* joerg.strebel@posteo.de
> *Cc:* user <user@spark.apache.org>
> *Subject:* Re: Watch "Airbus makes more of the sky with Spark - Jesse
> Anderson & Hassene Ben Salem" on YouTube
>
>
>
> My motive is simple . I want you (spark product  experts user)  to
> challenge the reason given by Jesse Anderson for choosing flink over spark.
>
>
>
> You know the saying keep your friends close, keep your enemies even closer.
>
> The video  only has  328 views.
>
> It is a great educational tool to see a recent recent Use Case. Should be
> of compelling interest to anyone in this field. Commercial Companies do not
> often share or discuss their projects openly.
>
>
>
> Incidentally Heathrow is the busiest airport in the world.
>
> 1. Because the emailing facility completed my sentence.
>
>
>
> 2. I think at Heathrow the gap is less than two minutes.
>
>
>
>
>
> On Sat, 25 Apr 2020, 09:42 Jörg Strebel, <joerg.strebel@gmail.com> wrote:
>
> Hallo!
>
> Well, the title of the video is actually "Airbus makes more of the sky
> with Flink - Jesse Anderson & Hassene Ben Salem"and it talks about Apache
> Flink and specifically not about Apache Spark.They excluded Spark Streaming
> for high latency reasons.
>
> Why are you posting this video on a Spark mailing list?
>
> Regards
>
> J. Strebel
>
> Am 25.04.20 um 05:07 schrieb Zahid Rahman:
>
>
>
> https://youtu.be/sYlbD_OoHhs
>
>
> Backbutton.co.uk
>
> ¯\_(ツ)_/¯
> ♡۶Java♡۶RMI ♡۶
>
> Make Use Method {MUM}
>
> makeuse.org
>
> --
>
> Jörg Strebel
>
> Aachener Straße 2 <https://www.google.com/maps/search/Aachener+Stra%C3%9Fe+2+80804+M%C3%BCnchen?entry=gmail&source=g>
>
> 80804 München <https://www.google.com/maps/search/Aachener+Stra%C3%9Fe+2+80804+M%C3%BCnchen?entry=gmail&source=g>
>
> --
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9  <https://amzn.to/2MaRAG9>
YouTube Live Streams: https://www.youtube.com/user/holdenkarau

Mime
View raw message