spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephen Carman <scar...@coldlight.com>
Subject Re: Tungsten + Flink
Date Fri, 01 May 2015 13:54:48 GMT
I think as long as the two frameworks follow the same paradigm for how their interfaces work
it’s fine to have 2 competing frameworks. This way the frameworks have some motivation
to be the best at what they do rather than being the only choice whether you like it or not.
They also seem to have some differing opinions about how to do certain things leaving me to
believe
that the 2 projects exist mostly because of disagreements on fundamentals about how a system
such as this should be built and scaled out.

I think spark should definitely take what it can from these projects, but otherwise they should
remain separate projects going their own way.

Steve


> On Apr 29, 2015, at 8:01 PM, Sree V <sree_at_chess@yahoo.com.INVALID> wrote:
>
> I agree, Ewan.
> We should also look into combining both Flink and Spark into one.This eases the industry
adaptation instead.
>
> Thanking you.
>
> With Regards
> Sree
>
>
>     On Wednesday, April 29, 2015 3:21 AM, Ewan Higgs <ewan.higgs@ugent.be> wrote:
>
>
> Hi all,
> A quick question about Tungsten. The announcement of the Tungsten
> project is on the back of Hadoop Summit in Brussels where some of the
> Flink devs were giving talks [1] on how Flink manages memory using byte
> arrays and the like to avoid the overhead of all the Java types[2]. Is
> there an opportunity for code reuse here? Spark and Flink may have
> different needs in some respects, but they work fundamentally towards
> the same goal so I imagine there could be come worthwhile collaboration.
>
> -Ewan
>
> [1] http://2015.hadoopsummit.org/brussels/speaker/?speaker=MrtonBalassi
> http://2015.hadoopsummit.org/brussels/speaker/?speaker=AljoschaKrettek
>
> [2]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53741525
> https://flink.apache.org/news/2015/03/13/peeking-into-Apache-Flinks-Engine-Room.html
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>
>

This e-mail is intended solely for the above-mentioned recipient and it may contain confidential
or privileged information. If you have received it in error, please notify us immediately
and delete the e-mail. You must not copy, distribute, disclose or take any action in reliance
on it. In addition, the contents of an attachment to this e-mail may contain software viruses
which could damage your own computer system. While ColdLight Solutions, LLC has taken every
reasonable precaution to minimize this risk, we cannot accept liability for any damage which
you sustain as a result of software viruses. You should perform your own virus checks before
opening the attachment.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org

Mime
View raw message