Thanks Sean! Let me take a look!

Iman

On Nov 8, 2016 7:=
29 AM, "Sean Owen" <sowe=
n@cloudera.com> wrote:

I think the problem here is that IndexedRow= Matrix.toRowMatrix does *not* result in a RowMatrix with rows in order of t= heir indices, necessarily:

// Drop its row indices.

= RowMatrix rowMat =3D indexedRowMatrix.toRowMatrix();

What = you get is a matrix where the rows are arranged in whatever order they were= passed to IndexedRowMatrix. RowMatrix says it's for rows where the ord= ering doesn't matter, but then it's maybe surprising it has a QR de= composition method, because clearly the result depends on the order of rows= in the input. (CC Yuhao Yang for a comment?)You = could say, well, why doesn't IndexedRowMatrix.toRowMatrix return at lea= st something with sorted rows? that would not be hard. It also won't re= turn "missing" rows (all zeroes), so it would not in any event re= sult in a RowMatrix whose implicit rows and ordering represented the same m= atrix. That, at least, strikes me as something to be better documented.=C2= =A0Maybe it would be nicer still to at least sort= the rows, given the existence of use cases like yours. For example, at lea= st CoordinateMatrix.toIndexedRowMatrix could sort? that is less surpri= sing. In any event you should be able to make it w= ork by manually getting the RDD[IndexedRow] out of IndexedRowMatrix, sortin= g by index, then mapping it to Vectors and making a RowMatrix from it.On= Tue, Nov 8, 2016 at 2:41 PM Iman Mohtashemi <iman.mohtashemi@gmail.com> wrot= e:Hi Sean,Here you go:sparsematrix.txt =3D=C2=A0row, col ,val0,0,.420,1,.28<= div class=3D"m_2764433045013519083gmail_msg">0,2,.891,0,.831,1,.341,2= ,.422,0,.233,0,.423,1,.983,2,.884,0,.23<= /div>4,1,.364,2,.97The vector is just the thir= d column of the matrix which should give the trivial solution of [0,0,1]This= translates to this which is correctThere are zeros in the matrix (Not really sparse but just an e= xample)0.42 =C2=A00.28 =C2=A00.89 =C2=A00.83 =C2=A00.34 =C2=A00.42 =C2=A00.23 =C2=A00.0 =C2=A0 0.0= =C2=A0=C2=A00.42 =C2= =A00.98 =C2=A00.88 =C2=A00.23 =C2=A00.36 =C2=A00.97 =C2=A0Here is what = I get for =C2=A0the Q and RQ: = -0.21470961288429483 =C2=A00.23590615093828807 =C2=A0 0.6784910613691661 = =C2=A0 =C2=A0-0.3920784= 235278427 =C2=A0 -0.06171221388256143 =C2=A00.5847874866876442 =C2=A0 =C2= =A0-0.7748216464954987 = =C2=A0 -0.4003560542230838 =C2=A0 -0.29392323671555354 =C2=A0-0.3920784235278427 =C2=A0 0.85179095= 21421976 =C2=A0 =C2=A0-0.31435038559403217 =C2=A0-0.21470961288429483 =C2=A0-0.23389547730301666 = =C2=A0-0.11165321782745863 =C2=A0R: -1.0712142642814275 =C2=A0-0.8347536340918976 =C2=A0-1.2276722= 25670157 =C2=A00.0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00.76628086911417= 17 =C2=A0 0.7553315911660984 =C2=A00.0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A00.0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00.7785= 210939368136 =C2=A0When running this in matlab the numbers are the sam= e but row 1 is the last row and the last row is interchanged with row 3On Mon, Nov = 7, 2016 at 11:35 PM Sean Owen <sowen@cloudera.com<= /a>> wrote:Rather than post a large section of cod= e, please post a small example of the input matrix and its decomposition, t= o illustrate what you're saying is out of order.