spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dongjoon Hyun (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-28086) Adds `random()` sql function
Date Mon, 29 Jul 2019 04:33:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-28086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16894922#comment-16894922
] 

Dongjoon Hyun commented on SPARK-28086:
---------------------------------------

This issue is reported twice at [~DylanGuedes]'s PR (https://github.com/apache/spark/pull/24881/files#diff-14489bae6b27814d4cde0456a7ae75c8R702)
and [~yumwang]'s PR (https://github.com/apache/spark/pull/25163/files#diff-23a3430e0e1ff88830cbb43701da1f2cR402).

For me, PostgreSQL random function is the same with Apache Spark `rand` as a uniform random
returning 0.0 <= x < 1.0.
- https://www.postgresql.org/docs/8.2/functions-math.html

Also, Spark also accepts `order by rand()` like the following.
{code}
spark-sql> SELECT rank() OVER (ORDER BY rank() OVER (ORDER BY rand()));
1
{code}

So, let's make an alias and unblock the other issues. I'll make a PR.

> Adds `random()` sql function
> ----------------------------
>
>                 Key: SPARK-28086
>                 URL: https://issues.apache.org/jira/browse/SPARK-28086
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>    Affects Versions: 3.0.0
>            Reporter: Dylan Guedes
>            Priority: Major
>
> Currently, Spark does not have a `random()` function. Postgres, however, does.
> For instance, this one is not valid:
> {code:sql}
> SELECT rank() OVER (ORDER BY rank() OVER (ORDER BY random()))
> {code}
> Because of the `random()` call. On the other hand, [Postgres has it.|https://www.postgresql.org/docs/8.2/functions-math.html]



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message