spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael Armbrust (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-11966) Spark API for UDTFs
Date Tue, 01 Dec 2015 02:08:10 GMT

    [ https://issues.apache.org/jira/browse/SPARK-11966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15032925#comment-15032925
] 

Michael Armbrust commented on SPARK-11966:
------------------------------------------

Ah, I was proposing the DataFrame function explode as it gives you something very close to
UDTFs.  However, if you want to be able to use the functions in pure SQL then thats not going
to be sufficient.

> Spark API for UDTFs
> -------------------
>
>                 Key: SPARK-11966
>                 URL: https://issues.apache.org/jira/browse/SPARK-11966
>             Project: Spark
>          Issue Type: New Feature
>          Components: SQL
>            Reporter: Jaka Jancar
>            Priority: Minor
>
> Defining UDFs is easy using sqlContext.udf.register, but not table-generating functions.
For those you still have to use these horrendous Hive interfaces:
> https://github.com/prongs/apache-hive/blob/master/contrib/src/java/org/apache/hadoop/hive/contrib/udtf/example/GenericUDTFCount2.java



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message