spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Russell Spitzer <russell.spit...@gmail.com>
Subject Re: Issue With mod function in Spark SQL
Date Tue, 17 Dec 2019 18:12:50 GMT
Is there a chance your data is all even or all odd?

On Tue, Dec 17, 2019 at 11:01 AM Tzahi File <tzahi.file@ironsrc.com> wrote:

> I have in my spark sql query a calculated field that gets the value if
> field1 % 3.
>
> I'm using this field as a partition so I expected to get 3 partitions in
> the mentioned case, and I do get. The issue happened with even numbers
> (instead of 3 - 4,2 ... ).
> When I tried to use even numbers, for example 4 I got only 2 partitions -
> 1 and 3.
> Field 1 datatype is bigint.
>
> Do you have any suggestions?
>
>
> --
> thanks,
> Tzahi
>

Mime
View raw message