spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matei Zaharia <>
Subject Draft ASF board report for November
Date Tue, 10 Nov 2020 20:00:08 GMT
Hi all,

It’s time to send in our quarterly ASF board report on Nov 11, so I wanted to include anything
notable going on that we want to appear in the board archive. Here is my draft; let me know
if you have suggested changes.


Apache Spark is a fast and general engine for large-scale data processing. It offers high-level
APIs in Java, Scala, Python, R and SQL as well as a rich set of libraries including stream
processing, machine learning, and graph analytics.

Project status:

- We releases Apache Spark 3.0.1 on September 8th and Spark 2.4.7 on September 12th as maintenance
releases with bug fixes to these two branches.

- The community is working on a number of new features in the Spark 3.x branch, including
improved data catalog APIs, a push-based shuffle implementation, and better error messages
to make Spark applications easier to debug. The largest changes have are being discussed as
SPIPs on our mailing list.


- One of the two software projects we reached out to July to change its name due to a trademark
issue has changed it. We are still waiting for a reply from the other one, but it may be that
development there has stopped.

Latest releases:

- Spark 2.4.7 was released on September 12th, 2020.
- Spark 3.0.1 was released on September 8th, 2020.
- Spark 3.0.0 was released on June 18th, 2020.

Committers and PMC:

- The latest committers were added on July 14th, 2020 (Huaxin Gao, Jungtaek Lim and Dilip
- The latest PMC member was added on Sept 4th, 2019 (Dongjoon Hyun). The PMC has been discussing
some new candidates to add as PMC members.
To unsubscribe e-mail:

View raw message