spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <so...@cloudera.com>
Subject Re: Apache Flink
Date Mon, 18 Apr 2016 06:46:32 GMT
Interesting tangent. I think there will never be a time when an
interesting area is covered only by one project, or product. Why are
there 30 SQL engines? or 50 car companies? it's a feature not a bug.
To the extent they provide different tradeoffs or functionality,
they're not entirely duplicative; to the extent they compete directly,
it's a win for the user.

As others have said, Flink (née Stratosphere) started quite a while
ago. But you can draw lines of influence back earlier than Spark. I
presume MS Dryad is the forerunner of all these.

And in case you wanted a third option, Google's DataFlow (now Apache
Beam) is really a reinvention of FlumeJava (nothing to do with Apache
Flume) from Google, in a way that Crunch was a port and minor update
of FlumeJava earlier. And it claims to run on Flink/Spark if you want.

https://cloud.google.com/dataflow/blog/dataflow-beam-and-spark-comparison


On Sun, Apr 17, 2016 at 10:25 PM, Mich Talebzadeh
<mich.talebzadeh@gmail.com> wrote:
> Also it always amazes me why they are so many tangential projects in Big
> Data space? Would not it be easier if efforts were spent on adding to Spark
> functionality rather than creating a new product like Flink?

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message