spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mucks17 <max-b...@web.de>
Subject Spark SQL: Assigning several aliases to the output (several return values) of an UDF
Date Mon, 19 Jan 2015 14:51:04 GMT
Hello



I use Hive on Spark and have an issue with assigning several aliases to the
output (several return values) of an UDF. I ran in several issues and ended
up with a workaround (described at the end of this message).

- Is assigning several aliases to the output of an UDF not supported by
Spark SQL yet?

- Is there a smarter solution than the one I ended up with finally - see
(3)?



1)

The query with following syntax is rejected due to the assigment of multiple
aliases.

Query
--------
SELECT my_function(param_one, param_two) AS (return_one, return_two,
return_three)
FROM my_table;

Error
--------
Unsupported language features in query: SELECT my_function(param_one,
param_two) AS (return_one, return_two, return_three)
FROM my_table;

TOK_QUERY
  TOK_FROM
    TOK_TABREF
      TOK_TABNAME
        my_table
    TOK_SELECT
      TOK_SELEXPR
        TOK_FUNCTION
          my_function
          TOK_TABLE_OR_COL
            param_one
          TOK_TABLE_OR_COL
            param_two
        return_one
        return_two
        return_three



2)

Because of this error I searched for a way to avoid assigning multiple
aliases to the UDF. I ended up having the following query and encountered
another error/issue.
Note: This error only occurs when having "c_0" in select clause. Only
selecting "c_1" and "c_2" works fine.

Query
--------
SELECT return.c_0 AS return_one, return.c_1 AS return_two, return.c_2 AS
return_three FROM (SELECT my_function(param_one, param_two) FROM my_table)
return;

Error
--------
java.lang.RuntimeException: Couldn't find c_0#504 in
[c_0#521L,c_1#522,c_2#523]



3)

My final (working) workaround is wrapping the actual query (the one with the
UDF) with an additional select statement.

Query
--------
SELECT result.c_0 AS return_one, result.c_1 AS return_two, result.c_2 AS
return_three FROM(SELECT * FROM (SELECT my_function(param_one, param_two)
FROM my_table) return) result;

Error
--------
No error :)



Kind regards
Max



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-Assigning-several-aliases-to-the-output-several-return-values-of-an-UDF-tp21236.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message