spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikael Ståldal <mikael.stal...@magine.com>
Subject Load whole ALS MatrixFactorizationModel into memory
Date Wed, 02 Nov 2016 16:53:57 GMT
import org.apache.spark.mllib.recommendation.ALS
import org.apache.spark.mllib.recommendation.MatrixFactorizationModel


I build a MatrixFactorizationModel with ALS.trainImplicit(), then I save it
with its save method.

Later, in an other process on another machine, I load the model with
MatrixFactorizationModel.load(). Now I want to make a lot of
recommendProducts() calls on the loaded model, and I want them to be quick,
without any I/O. However, they are slow and seem to to I/O each time.

Is there any way to force loading the whole model into memory (that step
can take some time and do I/O) and then be able to do recommendProducts()
on it multiple times, quickly without I/O?

-- 
[image: MagineTV]

*Mikael Ståldal*
Senior software developer

*Magine TV*
mikael.staldal@magine.com
Grev Turegatan 3  | 114 46 Stockholm, Sweden  |   www.magine.com

Privileged and/or Confidential Information may be contained in this
message. If you are not the addressee indicated in this message
(or responsible for delivery of the message to such a person), you may not
copy or deliver this message to anyone. In such case,
you should destroy this message and kindly notify the sender by reply
email.

Mime
View raw message