spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Will Benton <wi...@redhat.com>
Subject odd test suite failures while adding functions to Catalyst
Date Wed, 09 Jul 2014 02:58:11 GMT
Hi all,

I was testing an addition to Catalyst today (reimplementing a Hive UDF) and ran into some
odd failures in the test suite.  In particular, it seems that what most of these have in common
is that an array is spuriously reversed somewhere.  For example, the stddev tests in the HiveCompatibilitySuite
all failed this way (note reversed synonyms list; note also that stddev isn't the function
I reimplemented):

   [info] - udf_std *** FAILED ***
   [info]   Results do not match for udf_std:
   [info]   DESCRIBE FUNCTION EXTENDED std
   [info]   == Logical Plan ==
   [info]   NativeCommand DESCRIBE FUNCTION EXTENDED std
   [info]   
   [info]   == Optimized Logical Plan ==
   [info]   NativeCommand DESCRIBE FUNCTION EXTENDED std
   [info]   
   [info]   == Physical Plan ==
   [info]   NativeCommand DESCRIBE FUNCTION EXTENDED std, [result#38637:0]
   [info]   result
   [info]   !== HIVE - 2 row(s) ==                                         == CATALYST - 2
row(s) ==
   [info]    std(x) - Returns the standard deviation of a set of numbers   std(x) - Returns
the standard deviation of a set of numbers
   [info]   !Synonyms: stddev_pop, stddev                                  Synonyms: stddev,
stddev_pop (HiveComparisonTest.scala:372)

I also saw a reversed array (relative to the expected array) of Hive settings in HiveQuerySuite.

I'll probably be able to track down where things are going wrong after getting away from my
desk for a bit, but I thought I'd send it out to the dev list in case this looks familiar
to anyone.  Has anyone seen this kind of failure before?



thanks,
wb

Mime
View raw message