spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dhanesh Padmanabhan <dhanesh12...@gmail.com>
Subject Re: how to retain part of the features in LogisticRegressionModel (spark2.0)
Date Sun, 19 Mar 2017 11:08:22 GMT
binomial. Please use in combination with onevsrest for multi-class problems
in spark 2.0.2

Dhanesh
+91-9741125245

On Sun, Mar 19, 2017 at 4:29 PM, jinhong lu <lujinhong2@gmail.com> wrote:

> By the way, I found in spark 2.1 I can use setFamily() to decide binomial
> or multinomial, but how  can I do the same thing in spark 2.0.2?
> If  not support , which one is used in spark 2.0.2?  binomial or
> multinomial?
>
> 在 2017年3月19日,18:12,jinhong lu <lujinhong2@gmail.com> 写道:
>
>
> I train my LogisticRegressionModel like this,  I want my model to retain
> only some of the features(e.g. 500 of them), not all the 5555 features.
> What shou I do?
> I use .setElasticNetParam(1.0), but still all the features is
> in lrModel.coefficients.
>
>  import org.apache.spark.ml.classification.LogisticRegression
>  val data=spark.read.format("libsvm").option("numFeatures",
> "5555").load("/tmp/data/training_data3")
>  val Array(trainingData, testData) = data.randomSplit(Array(0.5, 0.5),
> seed = 1234L)
>
>  val lr = new LogisticRegression()
>  val lrModel = lr.fit(trainingData)
>  println(s"Coefficients: ${lrModel.coefficients} Intercept:
> ${lrModel.intercept}")
>
>  val predictions = lrModel.transform(testData)
>  predictions.show()
>
>
> Thanks,
> lujinhong
>
>
> Thanks,
> lujinhong
>
>

Mime
View raw message