spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shahab <shahab.mok...@gmail.com>
Subject Re: Registering custom UDAFs with HiveConetxt in SparkSQL, how?
Date Tue, 10 Mar 2015 17:43:48 GMT
Thanks Hao,
But my question concerns UDAF (user defined aggregation function ) not
UDTF( user defined type function ).
I appreciate if you could point me to some starting point on UDAF
development in Spark.

Thanks
Shahab

On Tuesday, March 10, 2015, Cheng, Hao <hao.cheng@intel.com> wrote:

>  Currently, Spark SQL doesn’t provide interface for developing the custom
> UDTF, but it can work seamless with Hive UDTF.
>
>
>
> I am working on the UDTF refactoring for Spark SQL, hopefully will provide
> an Hive independent UDTF soon after that.
>
>
>
> *From:* shahab [mailto:shahab.mokari@gmail.com
> <javascript:_e(%7B%7D,'cvml','shahab.mokari@gmail.com');>]
> *Sent:* Tuesday, March 10, 2015 5:44 PM
> *To:* user@spark.apache.org
> <javascript:_e(%7B%7D,'cvml','user@spark.apache.org');>
> *Subject:* Registering custom UDAFs with HiveConetxt in SparkSQL, how?
>
>
>
> Hi,
>
>
>
> I need o develop couple of UDAFs and use them in the SparkSQL. While UDFs
> can be registered as a function in HiveContext, I could not find any
> documentation of how UDAFs can be registered in the HiveContext?? so far
> what I have found is to make a JAR file, out of developed UDAF class, and
> then deploy the JAR file to SparkSQL .
>
>
>
> But is there any way to avoid deploying the jar file and register it
> programmatically?
>
>
>
>
>
> best,
>
> /Shahab
>

Mime
View raw message