Thank you Fred! We are actually doing what you suggested. We were concerned about the large
"null" matrix we are generating at the beginning of the process. But if large zero matrix
is not a performance issue, then I guess we are on the right track.
Thanks a lot!
Arijit
________________________________
From: Frederick R Reiss <frreiss@us.ibm.com>
Sent: Wednesday, May 3, 2017 5:14 AM
To: dev@systemml.incubator.apache.org
Subject: Re: Combining Multiple Matrix
Hi Arijit,
Have you considered using parfor and left indexing? Here's an example from the docs that generates
a large matrix out of smaller chunks:
ms = matrix(0, rows=2, cols=3*10)
parfor (v in 1:10, check=0) {
mv = matrix(v, rows=2, cols=3)
ms[,(v1)*3+1:v*3] = mv
}
SystemML has a special data structure to represent a matrix block that is all zeros, so creating
a very large matrix of zeros (first line in the above listing) is an inexpensive operation.
Fred
[Inactive hide details for arijit chakraborty 05/02/2017 09:48:13 AMHi, Our process
is generating multiple matrix, of same]arijit chakraborty 05/02/2017 09:48:13 AMHi,
Our process is generating multiple matrix, of same size, but number of matrix is random. Finally
From: arijit chakraborty <akc14@hotmail.com>
To: "dev@systemml.incubator.apache.org" <dev@systemml.incubator.apache.org>
Date: 05/02/2017 09:48 AM
Subject: Combining Multiple Matrix
________________________________
Hi,
Our process is generating multiple matrix, of same size, but number of matrix is random. Finally,
we combine all the smaller matrix to get a final large matrix.
The way we are creating the large matrix is, we are creating a blank matrix and then update
the matrix once each smaller matrix is created. This is inefficient because we are initially
creating a large matrix, which sometimes take more than required space.
So is there any method where we can combined all the final smaller matrix together into a
larger matrix? For rbind and cbind, we have to use loops to merge the matrix.
Thank you!
Arijit
