spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mucks17 <max-b...@web.de>
Subject Spark SQL: Assigning several aliases to the output (several return values) of an UDF
Date Mon, 19 Jan 2015 16:26:37 GMT
Hello 



I use Hive on Spark and have an issue with assigning several aliases to the
output (several return values) of an UDF. I ran in several issues and ended
up with a workaround (described at the end of this message). 

- Is assigning several aliases to the output of an UDF not supported by
Spark SQL yet? 

- Is there a smarter solution than the one I ended up with finally - see
(3)? 



1) 

The query with following syntax is rejected due to the assigment of multiple
aliases. 

Query 
-------- 
SELECT my_function(param_one, param_two) AS (return_one, return_two,
return_three) 
FROM my_table; 

Error 
-------- 
Unsupported language features in query: SELECT my_function(param_one,
param_two) AS (return_one, return_two, return_three) 
FROM my_table; 

TOK_QUERY 
  TOK_FROM 
    TOK_TABREF 
      TOK_TABNAME 
        my_table 
    TOK_SELECT 
      TOK_SELEXPR 
        TOK_FUNCTION 
          my_function 
          TOK_TABLE_OR_COL 
            param_one 
          TOK_TABLE_OR_COL 
            param_two 
        return_one 
        return_two 
        return_three 



2) 

Because of this error I searched for a way to avoid assigning multiple
aliases to the UDF. I ended up having the following query and encountered
another error/issue. 
Note: This error only occurs when having "c_0" in select clause. Only
selecting "c_1" and "c_2" works fine. 

Query 
-------- 
SELECT return.c_0 AS return_one, return.c_1 AS return_two, return.c_2 AS
return_three FROM (SELECT my_function(param_one, param_two) FROM my_table)
return; 

Error 
-------- 
java.lang.RuntimeException: Couldn't find c_0#504 in
[c_0#521L,c_1#522,c_2#523] 



3) 

My final (working) workaround is wrapping the actual query (the one with the
UDF) with an additional select statement. 

Query 
-------- 
SELECT result.c_0 AS return_one, result.c_1 AS return_two, result.c_2 AS
return_three FROM(SELECT * FROM (SELECT my_function(param_one, param_two)
FROM my_table) return) result; 

Error 
-------- 
No error :) 



Kind regards 
Max



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-Assigning-several-aliases-to-the-output-several-return-values-of-an-UDF-tp21238.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org


Mime
View raw message