Wait...it seem SparkContext does not provide a way to save/load object files. It can only save/load RDD. What do I missed here?


Thanks,
David


On Sat, Mar 7, 2015 at 11:05 PM Xi Shen <davidshen84@gmail.com> wrote:
Ah~it is serializable. Thanks!


On Sat, Mar 7, 2015 at 10:59 PM Ekrem Aksoy <ekremaksoy@gmail.com> wrote:
You can serialize your trained model to persist somewhere.

Ekrem Aksoy

On Sat, Mar 7, 2015 at 12:10 PM, Xi Shen <davidshen84@gmail.com> wrote:
Hi,

I checked a few ML algorithms in MLLib.


I could not find a way to save the trained model. Does this means I have to train my model every time? Is there a more economic way to do this?

I am thinking about something like:

model.run(...)
model.save("hdfs://path/to/hdfs")

Then, next I can do:

val model = Model.createFrom("hdfs://...")
model.predict(vector)

I am new to spark, maybe there are other ways to persistent the model?


Thanks,
David