spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cheng Lian <lian.cs....@gmail.com>
Subject Re: Spark SQL: Assigning several aliases to the output (several return values) of an UDF
Date Tue, 20 Jan 2015 20:12:37 GMT
Guess this can be helpful: 
http://stackoverflow.com/questions/14252615/stack-function-in-hive-how-to-specify-multiple-aliases

On 1/19/15 8:26 AM, mucks17 wrote:
> Hello
>
>
>
> I use Hive on Spark and have an issue with assigning several aliases to the
> output (several return values) of an UDF. I ran in several issues and ended
> up with a workaround (described at the end of this message).
>
> - Is assigning several aliases to the output of an UDF not supported by
> Spark SQL yet?
>
> - Is there a smarter solution than the one I ended up with finally - see
> (3)?
>
>
>
> 1)
>
> The query with following syntax is rejected due to the assigment of multiple
> aliases.
>
> Query
> --------
> SELECT my_function(param_one, param_two) AS (return_one, return_two,
> return_three)
> FROM my_table;
>
> Error
> --------
> Unsupported language features in query: SELECT my_function(param_one,
> param_two) AS (return_one, return_two, return_three)
> FROM my_table;
>
> TOK_QUERY
>    TOK_FROM
>      TOK_TABREF
>        TOK_TABNAME
>          my_table
>      TOK_SELECT
>        TOK_SELEXPR
>          TOK_FUNCTION
>            my_function
>            TOK_TABLE_OR_COL
>              param_one
>            TOK_TABLE_OR_COL
>              param_two
>          return_one
>          return_two
>          return_three
>
>
>
> 2)
>
> Because of this error I searched for a way to avoid assigning multiple
> aliases to the UDF. I ended up having the following query and encountered
> another error/issue.
> Note: This error only occurs when having "c_0" in select clause. Only
> selecting "c_1" and "c_2" works fine.
>
> Query
> --------
> SELECT return.c_0 AS return_one, return.c_1 AS return_two, return.c_2 AS
> return_three FROM (SELECT my_function(param_one, param_two) FROM my_table)
> return;
>
> Error
> --------
> java.lang.RuntimeException: Couldn't find c_0#504 in
> [c_0#521L,c_1#522,c_2#523]
>
>
>
> 3)
>
> My final (working) workaround is wrapping the actual query (the one with the
> UDF) with an additional select statement.
>
> Query
> --------
> SELECT result.c_0 AS return_one, result.c_1 AS return_two, result.c_2 AS
> return_three FROM(SELECT * FROM (SELECT my_function(param_one, param_two)
> FROM my_table) return) result;
>
> Error
> --------
> No error :)
>
>
>
> Kind regards
> Max
>
>
>
> --
> View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-Assigning-several-aliases-to-the-output-several-return-values-of-an-UDF-tp21238.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message