spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Siegmann <>
Subject Re: SparkSQL production readiness
Date Mon, 02 Mar 2015 23:22:28 GMT
OK, good to know data frames are still experimental. Thanks Michael.

On Mon, Mar 2, 2015 at 12:37 PM, Michael Armbrust <>

> We have been using Spark SQL in production for our customers at Databricks
> for almost a year now.  We also know of some very large production
> deployments elsewhere.  It is still a young project, but I wouldn't call it
> alpha.
> The primary changes to the API are the addition of the DataFrame
> interface, which is an expansion of the DSL that was already there.  All of
> the SQL / HiveQL stuff remains unchanged, as well as the internal execution
> engine.  DataFrames are still marked experimental, since as you said, we
> should let people use them before cementing them.

View raw message