spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Cutler <cutl...@gmail.com>
Subject Re: ML:One vs Rest with crossValidator for multinomial in logistic regression
Date Fri, 09 Feb 2018 01:17:13 GMT
Nicolas, are you referring to printing the model params in that example
with "print(model1.extractParamMap())"?  There was a problem with pyspark
models not having params after being fit, which causes this example to show
nothing for model paramMaps.  This was fixed in
https://issues.apache.org/jira/browse/SPARK-10931 and the example now shows
all model params.  The fix will be in the Spark 2.3 release.

Bryan

On Wed, Jan 31, 2018 at 10:20 PM, Nicolas Paris <niparisco@gmail.com> wrote:

> Hey
>
> I am also interested in how to get those parameters.
> For example, the demo code spark-2.2.1-bin-hadoop2.7/
> examples/src/main/python/ml/estimator_transformer_param_example.py
> return empty parameters when  printing "lr.extractParamMap()"
>
> That's weird
>
> Thanks
>
> Le 30 janv. 2018 à 23:10, Bryan Cutler écrivait :
> > Hi Michelle,
> >
> > Your original usage of ParamGridBuilder was not quite right, `addGrid`
> expects
> > (some parameter, array of values for that parameter).  If you want to do
> a grid
> > search with different regularization values, you would do the following:
> >
> > val paramMaps = new ParamGridBuilder().addGrid(logist.regParam,
> Array(0.1,
> > 0.3)).build()
> >
> > * don't forget to build the grid after adding values
> >
> > On Tue, Jan 30, 2018 at 6:55 AM, michelleyang <
> michelle1026shuyu@gmail.com>
> > wrote:
> >
> >     I tried to use One vs Rest in spark ml with pipeline and
> crossValidator for
> >     multimultinomial in logistic regression.
> >
> >     It came out with empty coefficients. I figured out it was the
> setting of
> >     ParamGridBuilder. Can anyone help me understand how does the
> parameter
> >     setting affect the crossValidator process?
> >
> >     the orginal code: //output empty coefficients.
> >
> >     val logist=new LogisticRegression
> >
> >     val ova = new OneVsRest().setClassifier(logist)
> >
> >     val paramMaps = new ParamGridBuilder().addGrid(ova.classifier,
> >     Array(logist.getRegParam))
> >
> >     New code://output multi classes coefficients
> >
> >     val logist=new LogisticRegression
> >
> >     val ova = new OneVsRest().setClassifier(logist)
> >
> >     val classifier1 = new LogisticRegression().setRegParam(2.0)
> >
> >     val classifier2 = new LogisticRegression().setRegParam(3.0)
> >
> >     val paramMaps = new ParamGridBuilder() .addGrid(ova.classifier,
> >     Array(classifier1, classifier2))
> >
> >     Please help!!!! Thanks.
> >
> >
> >
> >     --
> >     Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
> >
> >     ------------------------------------------------------------
> ---------
> >     To unsubscribe e-mail: user-unsubscribe@spark.apache.org
> >
> >
> >
>

Mime
View raw message