spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Armbrust <mich...@databricks.com>
Subject Re: SparkSQL production readiness
Date Mon, 02 Mar 2015 17:37:18 GMT
We have been using Spark SQL in production for our customers at Databricks
for almost a year now.  We also know of some very large production
deployments elsewhere.  It is still a young project, but I wouldn't call it
alpha.

The primary changes to the API are the addition of the DataFrame interface,
which is an expansion of the DSL that was already there.  All of the SQL /
HiveQL stuff remains unchanged, as well as the internal execution engine.
DataFrames are still marked experimental, since as you said, we should let
people use them before cementing them.

On Mon, Mar 2, 2015 at 7:29 AM, Daniel Siegmann <daniel.siegmann@teamaol.com
> wrote:

> I thought removing the alpha tag just meant the API was stable? Speaking
> of which, aren't there major changes to the API coming in 1.3? Why are you
> marking the API as stable before these changes have been widely used?
>
>
> On Sat, Feb 28, 2015 at 5:17 PM, Michael Armbrust <michael@databricks.com>
> wrote:
>
>> We are planning to remove the alpha tag in 1.3.0.
>>
>> On Sat, Feb 28, 2015 at 12:30 AM, Wang, Daoyuan <daoyuan.wang@intel.com>
>> wrote:
>>
>>>  Hopefully  the alpha tag will be remove in 1.4.0, if the community can
>>> review code a little bit faster :P
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Daoyuan
>>>
>>>
>>>
>>> *From:* Ashish Mukherjee [mailto:ashish.mukherjee@gmail.com]
>>> *Sent:* Saturday, February 28, 2015 4:28 PM
>>> *To:* user@spark.apache.org
>>> *Subject:* SparkSQL production readiness
>>>
>>>
>>>
>>> Hi,
>>>
>>>
>>>
>>> I am exploring SparkSQL for my purposes of performing large relational
>>> operations across a cluster. However, it seems to be in alpha right now. Is
>>> there any indication when it would be considered production-level? I don't
>>> see any info on the site.
>>>
>>>
>>>
>>> Regards,
>>>
>>> Ashish
>>>
>>
>>
>

Mime
View raw message