Thanks Sean! Let me take a look!
Iman

On Nov 8, 2016 7:= 29 AM, "Sean Owen" <sowe= n@cloudera.com> wrote:
I think the problem here is that IndexedRow= Matrix.toRowMatrix does *not* result in a RowMatrix with rows in order of t= heir indices, necessarily:

// Drop its row indices.
= RowMatrix rowMat =3D indexedRowMatrix.toRowMatrix();

What = you get is a matrix where the rows are arranged in whatever order they were= passed to IndexedRowMatrix. RowMatrix says it's for rows where the ord= ering doesn't matter, but then it's maybe surprising it has a QR de= composition method, because clearly the result depends on the order of rows= in the input. (CC Yuhao Yang for a comment?)

You = could say, well, why doesn't IndexedRowMatrix.toRowMatrix return at lea= st something with sorted rows? that would not be hard. It also won't re= turn "missing" rows (all zeroes), so it would not in any event re= sult in a RowMatrix whose implicit rows and ordering represented the same m= atrix. That, at least, strikes me as something to be better documented.=C2= =A0

Maybe it would be nicer still to at least sort= the rows, given the existence of use cases like yours. For example, at lea= st CoordinateMatrix.toIndexedRowMatrix could sort? that is less surpri= sing.

In any event you should be able to make it w= ork by manually getting the RDD[IndexedRow] out of IndexedRowMatrix, sortin= g by index, then mapping it to Vectors and making a RowMatrix from it.

On= Tue, Nov 8, 2016 at 2:41 PM Iman Mohtashemi <iman.mohtashemi@gmail.com> wrot= e:
Hi Sean,
Here you go:

sparsematrix.txt =3D=C2=A0

row, col ,val
0,0,.42
0,1,.28
<= div class=3D"m_2764433045013519083gmail_msg">0,2,.89
1,0,.83
1,1,.34
1,2= ,.42
2,0,.23
3,0,.42
3,1,.98
3,2,.88
4,0,.23<= /div>
4,1,.36
4,2,.97

The vector is just the thir= d column of the matrix which should give the trivial solution of [0,0,1]

This= translates to this which is correct
There are zeros in the matrix (Not really sparse but just an e= xample)
0.42 =C2=A00.28 =C2=A00.89 =C2=A0
0.83 =C2=A00.34 =C2=A00.42 =C2=A0
0.23 =C2=A00.0 =C2=A0 0.0= =C2=A0=C2=A0
0.42 =C2= =A00.98 =C2=A00.88 =C2=A0
0.23 =C2=A00.36 =C2=A00.97 =C2=A0

Here is what = I get for =C2=A0the Q and R

Q: = -0.21470961288429483 =C2=A00.23590615093828807 =C2=A0 0.6784910613691661 = =C2=A0 =C2=A0
-0.3920784= 235278427 =C2=A0 -0.06171221388256143 =C2=A00.5847874866876442 =C2=A0 =C2= =A0
-0.7748216464954987 = =C2=A0 -0.4003560542230838 =C2=A0 -0.29392323671555354 =C2=A0
-0.3920784235278427 =C2=A0 0.85179095= 21421976 =C2=A0 =C2=A0-0.31435038559403217 =C2=A0
-0.21470961288429483 =C2=A0-0.23389547730301666 = =C2=A0-0.11165321782745863 =C2=A0
R: -1.0712142642814275 =C2=A0-0.8347536340918976 =C2=A0-1.2276722= 25670157 =C2=A0
0.0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00.76628086911417= 17 =C2=A0 0.7553315911660984 =C2=A0
0.0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A00.0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A00.7785= 210939368136 =C2=A0

When running this in matlab the numbers are the sam= e but row 1 is the last row and the last row is interchanged with row 3

On Mon, Nov = 7, 2016 at 11:35 PM Sean Owen <sowen@cloudera.com<= /a>> wrote:
--001a1140f03814a92f0540cbd290--