spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "DEVAN M.S." <msdeva...@gmail.com>
Subject Re: IF statement doesn't work in Spark-SQL?
Date Tue, 20 Jan 2015 11:15:50 GMT
Which context are you using HiveContext or SQLContext ? Can you try
with HiveContext
??


Devan M.S. | Research Associate | Cyber Security | AMRITA VISHWA
VIDYAPEETHAM | Amritapuri | Cell +919946535290 |


On Tue, Jan 20, 2015 at 3:49 PM, Xuelin Cao <xuelincao2014@gmail.com> wrote:

>
> Hi, I'm using Spark 1.2
>
>
> On Tue, Jan 20, 2015 at 5:59 PM, Wang, Daoyuan <daoyuan.wang@intel.com>
> wrote:
>
>>  Hi Xuelin,
>>
>>
>>
>> What version of Spark are you using?
>>
>>
>>
>> Thanks,
>>
>> Daoyuan
>>
>>
>>
>> *From:* Xuelin Cao [mailto:xuelincao2014@gmail.com]
>> *Sent:* Tuesday, January 20, 2015 5:22 PM
>> *To:* User
>> *Subject:* IF statement doesn't work in Spark-SQL?
>>
>>
>>
>>
>>
>> Hi,
>>
>>
>>
>>       I'm trying to migrate some hive scripts to Spark-SQL. However, I
>> found some statement is incompatible in Spark-sql.
>>
>>
>>
>>       Here is my SQL. And the same SQL works fine in HIVE environment.
>>
>>
>>
>>     SELECT
>>
>>       *if(ad_user_id>1000, 1000, ad_user_id) as user_id*
>>
>>     FROM
>>
>>       ad_search_keywords
>>
>>
>>
>>      What I found is, the parser reports error on the "*if*" statement:
>>
>>
>>
>> No function to evaluate expression. type: AttributeReference, tree:
>> ad_user_id#4
>>
>>
>>
>>
>>
>>      Anyone have any idea about this?
>>
>>
>>
>>
>>
>
>

Mime
View raw message