spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jörn Franke <jornfra...@gmail.com>
Subject Re: Is it okay to run Hive Java UDFS in Spark-sql. Anybody's still doing it?
Date Thu, 02 Feb 2017 09:14:05 GMT

There are many performance aspects here which may not only related to the UDF itself, but
on configuration of platform, data etc.

You seem to have a performance problem with your UDFs. Maybe you can elaborate on 
1) what data you process (format, etc)
2) what you try to Analyse
3) how you implemented your udfs. Maybe the implementation is not optimal and then simply
moving it from hive to spark does not give you any benefits. Bad code is still bad code in
SparkSql


> On 2 Feb 2017, at 09:33, Alex <siri8123@gmail.com> wrote:
> 
> Hi Team,
> 
> Do you really think if we make Hive Java UDF's to run on spark-sql  it will make performance
difference???  IS anybody here actually doing it.. converting Hive UDF's to run on Spark-sql..
> 
> What would be your approach if asked to make Hive Java UDFS project run on spark-sql
> 
> Would yu run the same java UDF using Spark-sql
>  
> or 
> 
> You would recode all java UDF to scala UDF and then run?
> 
> 
> Regards,
> Alex

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message