spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Khalid Mammadov <>
Subject Re: mission statement : unified
Date Sun, 25 Oct 2020 20:58:19 GMT
Correct. Also as explained in the book LearningSpark2.0 by Databiricks:

Unified Analytics
While the notion of unification is not unique to Spark, it is a core component of its design
philosophy and evolution. In November 2016, the Association for Computing Machinery (ACM)
recognized Apache Spark and conferred upon its original creators the prestigious ACM Award
for their paper describing Apache Spark as a “Unified Engine for Big Data Processing.”
The award-winning paper notes that Spark replaces all the separate batch processing, graph,
stream, and query engines like Storm, Impala, Dremel, Pregel, etc. with a unified stack of
components that addresses diverse workloads under a single distributed fast engine.


> On 19 Oct 2020, at 07:03, Sonal Goyal <> wrote:
> My thought is that Spark supports analytics for structured and unstructured data, batch
as well as real time. This was pretty revolutionary when Spark first came out. That's where
the unified term came from I think. Even after all these years, Spark remains the trusted
framework for enterprise analytics. 
>> On Mon, 19 Oct 2020, 11:24 Gourav Sengupta < wrote:
>> Hi,
>> I think that it is just a marketing statement. But with SPARK 3.x, now that you are
seeing that SPARK is no more than just another distributed data processing engine, they are
trying to join data pre-processing into ML pipelines directly. I may call that unified. 
>> But you get the same with several other frameworks as well now so not quite sure
how unified creates a unique brand value.
>> Regards,
>> Gourav Sengupta 
>>> On Sun, Oct 18, 2020 at 6:40 PM Hulio andres <> wrote:
>>> Apache Spark's  mission statement is  Apache Spark™ is a unified analytics
engine for large-scale data processing. 
>>> To what is the word "unified" inferring ?
>>> --------------------------------------------------------------------- To unsubscribe

View raw message