spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Hive UDAF percentile_approx says "This UDAF does not support the deprecated getEvaluator() method."
Date Tue, 13 Jan 2015 22:41:19 GMT
Looking at the source code for AbstractGenericUDAFResolver, the following
(non-deprecated) method should be called:

  public GenericUDAFEvaluator getEvaluator(GenericUDAFParameterInfo info)

It is called by hiveUdfs.scala (master branch):

    val parameterInfo = new
SimpleGenericUDAFParameterInfo(inspectors.toArray, false, false)
    resolver.getEvaluator(parameterInfo)

FYI

On Tue, Jan 13, 2015 at 1:51 PM, Jianshi Huang <jianshi.huang@gmail.com>
wrote:

> Hi,
>
> The following SQL query
>
> select percentile_approx(variables.var1, 0.95) p95
> from model
>
> will throw
>
> ERROR SparkSqlInterpreter: Error
> org.apache.hadoop.hive.ql.parse.SemanticException: This UDAF does not
> support the deprecated getEvaluator() method.
>         at
> org.apache.hadoop.hive.ql.udf.generic.AbstractGenericUDAFResolver.getEvaluator(AbstractGenericUDAFResolver.java:53)
>         at
> org.apache.spark.sql.hive.HiveGenericUdaf.objectInspector$lzycompute(hiveUdfs.scala:196)
>         at
> org.apache.spark.sql.hive.HiveGenericUdaf.objectInspector(hiveUdfs.scala:195)
>         at
> org.apache.spark.sql.hive.HiveGenericUdaf.dataType(hiveUdfs.scala:203)
>         at
> org.apache.spark.sql.catalyst.expressions.Alias.toAttribute(namedExpressions.scala:105)
>         at
> org.apache.spark.sql.catalyst.plans.logical.Aggregate$$anonfun$output$6.apply(basicOperators.scala:143)
>         at
> org.apache.spark.sql.catalyst.plans.logical.Aggregate$$anonfun$output$6.apply(basicOperators.scala:143)
>
> I'm using latest branch-1.2
>
> I found in PR that percentile and percentile_approx are supported. A bug?
>
> Thanks,
> --
> Jianshi Huang
>
> LinkedIn: jianshi
> Twitter: @jshuang
> Github & Blog: http://huangjs.github.com/
>

Mime
View raw message