spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Armbrust <mich...@databricks.com>
Subject Re: what is the roadmap for Spark SQL dialect in the coming releases?
Date Sun, 25 Jan 2015 20:48:00 GMT
Yeah, the HiveContext is just a SQLContext that is extended with HQL,
access to a metastore, hive UDFs and hive serdes.  The query execution
however is identical to a SQLContext.

On Sun, Jan 25, 2015 at 7:24 AM, Niranda Perera <niranda.perera@gmail.com>
wrote:

> Thanks Michael.
>
> A clarification. So the HQL dialect provided by HiveContext, does it use
> catalyst optimizer? I though HiveContext is only related to Hive
> integration in Spark!
>
> Would be grateful if you could clarify this
>
> cheers
>
> On Sun, Jan 25, 2015 at 1:23 AM, Michael Armbrust <michael@databricks.com>
> wrote:
>
>> I generally recommend people use the HQL dialect provided by the
>> HiveContext when possible:
>> http://spark.apache.org/docs/latest/sql-programming-guide.html#getting-started
>>
>> I'll also note that this is distinct from the Hive on Spark project,
>> which is based on the Hive query optimizer / execution engine instead of
>> the catalyst optimizer that is shipped with Spark.
>>
>> On Thu, Jan 22, 2015 at 3:12 AM, Niranda Perera <niranda.perera@gmail.com
>> > wrote:
>>
>>> Hi,
>>>
>>> would like to know if there is an update on this?
>>>
>>> rgds
>>>
>>> On Mon, Jan 12, 2015 at 10:44 AM, Niranda Perera <
>>> niranda.perera@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I found out that SparkSQL supports only a relatively small subset of
>>>> SQL dialect currently.
>>>>
>>>> I would like to know the roadmap for the coming releases.
>>>>
>>>> And, are you focusing more on popularizing the 'Hive on Spark' SQL
>>>> dialect or the Spark SQL dialect?
>>>>
>>>> Rgds
>>>> --
>>>> Niranda
>>>>
>>>
>>>
>>>
>>> --
>>> Niranda
>>>
>>
>>
>
>
> --
> Niranda
>

Mime
View raw message