spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "zhengruifeng (JIRA)" <>
Subject [jira] [Commented] (SPARK-18757) Models in Pyspark support column setters
Date Tue, 03 Jan 2017 02:56:58 GMT


zhengruifeng commented on SPARK-18757:

OK, I will follow your guides.

> Models in Pyspark support column setters
> ----------------------------------------
>                 Key: SPARK-18757
>                 URL:
>             Project: Spark
>          Issue Type: Brainstorming
>          Components: ML, PySpark
>            Reporter: zhengruifeng
> Recently, I found three places in which column setters are missing: KMeansModel, BisectingKMeansModel
and OneVsRestModel.
> These three models directly inherit `Model` which dont have columns setters, so I had
to add the missing setters manually in [SPARK-18625] and [SPARK-18520].
> Fow now, models in pyspark still don't support column setters at all.
> I suggest that we keep the hierarchy of pyspark models in line with that in the scala
> For classifiation and regression algs, I‘m making a trial in [SPARK-18739]. In it,
I try to copy the hierarchy from the scala side.
> For clustering algs, I think we may first create abstract classes {{ClusteringModel}}
and {{ProbabilisticClusteringModel}} in the scala side, and make clustering algs inherit it.
Then, in the python side, we copy the hierarchy so that we dont need to add setters manually
for each alg.
> For features algs, we can also use a abstract class {{FeatureModel}} in scala side, and
do the same thing.
> What's your opinions? [~yanboliang][~josephkb][~sethah][~srowen]

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message