spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yanbo Liang <yblia...@gmail.com>
Subject Re: Saving a pyspark.ml.feature.PCA model
Date Sun, 24 Jul 2016 11:51:24 GMT
Sorry for the wrong link, what you should refer is jpmml-sparkml (
https://github.com/jpmml/jpmml-sparkml).

Thanks
Yanbo

2016-07-24 4:46 GMT-07:00 Yanbo Liang <ybliang8@gmail.com>:

> Spark does not support exporting ML models to PMML currently. You can try
> the third party jpmml-spark (https://github.com/jpmml/jpmml-spark)
> package which supports a part of ML models.
>
> Thanks
> Yanbo
>
> 2016-07-20 11:14 GMT-07:00 Ajinkya Kale <kaleajinkya@gmail.com>:
>
>> Just found Google dataproc has a preview of spark 2.0. Tried it and
>> save/load works! Thanks Shuai.
>> Followup question - is there a way to export the pyspark.ml models to
>> PMML ? If not, what is the best way to integrate the model for inference in
>> a production service ?
>>
>> On Tue, Jul 19, 2016 at 8:22 PM Ajinkya Kale <kaleajinkya@gmail.com>
>> wrote:
>>
>>> I am using google cloud dataproc which comes with spark 1.6.1. So
>>> upgrade is not really an option.
>>> No way / hack to save the models in spark 1.6.1 ?
>>>
>>> On Tue, Jul 19, 2016 at 8:13 PM Shuai Lin <linshuai2012@gmail.com>
>>> wrote:
>>>
>>>> It's added in not-released-yet 2.0.0 version.
>>>>
>>>> https://issues.apache.org/jira/browse/SPARK-13036
>>>> https://github.com/apache/spark/commit/83302c3b
>>>>
>>>> so i guess you need to wait for 2.0 release (or use the current rc4).
>>>>
>>>> On Wed, Jul 20, 2016 at 6:54 AM, Ajinkya Kale <kaleajinkya@gmail.com>
>>>> wrote:
>>>>
>>>>> Is there a way to save a pyspark.ml.feature.PCA model ? I know mllib
>>>>> has that but mllib does not have PCA afaik. How do people do model
>>>>> persistence for inference using the pyspark ml models ? Did not find
any
>>>>> documentation on model persistency for ml.
>>>>>
>>>>> --ajinkya
>>>>>
>>>>
>>>>
>

Mime
View raw message