spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <sro...@gmail.com>
Subject Re: Ask for ARM CI for spark
Date Sat, 27 Jul 2019 13:04:27 GMT
Great thanks - we can take this to JIRAs now.
I think it's worth changing the implementation of atanh if the test value
just reflects what Spark does, and there's evidence is a little bit
inaccurate.
There's an equivalent formula which seems to have better accuracy.

On Fri, Jul 26, 2019 at 10:02 PM Takeshi Yamamuro <linguin.m.s@gmail.com>
wrote:

> Hi, all,
>
> FYI:
> >> @Yuming Wang the results in float8.sql are from PostgreSQL directly?
> >> Interesting if it also returns the same less accurate result, which
> >> might suggest it's more to do with underlying OS math libraries. You
> >> noted that these tests sometimes gave platform-dependent differences
> >> in the last digit, so wondering if the test value directly reflects
> >> PostgreSQL or just what we happen to return now.
>
> The results in float8.sql.out were recomputed in Spark/JVM.
> The expected output of the PostgreSQL test is here:
> https://github.com/postgres/postgres/blob/master/src/test/regress/expected/float8.out#L493
>
> As you can see in the file (float8.out), the results other than atanh
> also are different between Spark/JVM and PostgreSQL.
> For example, the answers of acosh are:
> -- PostgreSQL
>
> https://github.com/postgres/postgres/blob/master/src/test/regress/expected/float8.out#L487
> 1.31695789692482
>
> -- Spark/JVM
>
> https://github.com/apache/spark/blob/master/sql/core/src/test/resources/sql-tests/results/pgSQL/float8.sql.out#L523
> 1.3169578969248166
>
> btw, the PostgreSQL implementation for atanh just calls atanh in math.h:
>
> https://github.com/postgres/postgres/blob/master/src/backend/utils/adt/float.c#L2606
>
> Bests,
> Takeshi
>
>

Mime
View raw message