spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Phillip Henry <>
Subject Distributed matrices with column counts represented by Int (rather than Long)
Date Thu, 26 May 2016 16:06:05 GMT

I notice that some DistributedMatrix represent the number of columns with
an Int rather than a Long (RowMatrix etc). This limits the number of
columns to about 2 billion.

We're approaching that limit. What do people recommend we do to mitigate
the problem? Are there plans to use a larger data type as the trait
suggests it should be?



View raw message