spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jacek Laskowski <ja...@japila.pl>
Subject Re: Deploying ML Pipeline Model
Date Fri, 01 Jul 2016 17:23:44 GMT
Hi Rishabh,

I've just today had similar conversation about how to do a ML Pipeline
deployment and couldn't really answer this question and more because I
don't really understand the use case.

What would you expect from ML Pipeline model deployment? You can save
your model to a file by model.write.overwrite.save("model_v1").

model_v1
|-- metadata
|   |-- _SUCCESS
|   `-- part-00000
`-- stages
    |-- 0_regexTok_b4265099cc1c
    |   `-- metadata
    |       |-- _SUCCESS
    |       `-- part-00000
    |-- 1_hashingTF_8de997cf54ba
    |   `-- metadata
    |       |-- _SUCCESS
    |       `-- part-00000
    `-- 2_linReg_3942a71d2c0e
        |-- data
        |   |-- _SUCCESS
        |   |-- _common_metadata
        |   |-- _metadata
        |   `-- part-r-00000-2096c55a-d654-42b2-90d3-5a310101cba5.gz.parquet
        `-- metadata
            |-- _SUCCESS
            `-- part-00000

9 directories, 12 files

What would you like to have outside SparkContext? What's wrong with
using Spark? Just curious hoping to understand the use case better.
Thanks.

Pozdrawiam,
Jacek Laskowski
----
https://medium.com/@jaceklaskowski/
Mastering Apache Spark http://bit.ly/mastering-apache-spark
Follow me at https://twitter.com/jaceklaskowski


On Fri, Jul 1, 2016 at 12:54 PM, Rishabh Bhardwaj <rbnext29@gmail.com> wrote:
> Hi All,
>
> I am looking for ways to deploy a ML Pipeline model in production .
> Spark has already proved to be a one of the best framework for model
> training and creation, but once the ml pipeline model is ready how can I
> deploy it outside spark context ?
> MLlib model has toPMML method but today Pipeline model can not be saved to
> PMML. There are some frameworks like MLeap which are trying to abstract
> Pipeline Model and provide ML Pipeline Model deployment outside spark
> context,but currently they don't have most of the ml transformers and
> estimators.
> I am looking for related work going on this area.
> Any pointers will be helpful.
>
> Thanks,
> Rishabh.

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message