spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yanwei Wayne Zhang <>
Subject Invert large matrix
Date Thu, 29 Dec 2016 02:22:26 GMT
Hi all,

I have a matrix X stored as RDD[SparseVector] that is high dimensional, say 800 million rows
and 2 million columns, and more 95% of the entries are zero.

Is there a way to invert (X'X + eye) efficiently, where X' is the transpose of X and eye is
the identity matrix? I am thinking of using RowMatrix but not sure if it is feasible.

Any suggestion is highly appreciated.



View raw message