spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yanbo Liang <yblia...@gmail.com>
Subject Re: Trainning a spark ml linear regresion model fail after migrating from 1.5.2 to 1.6.1
Date Sat, 02 Jul 2016 08:19:13 GMT
Yes, WeightedLeastSquares can not solve some ill-conditioned problem
currently, the community members have paid some efforts to resolve it
(SPARK-13777). For the work around, you can set the solver to "l-bfgs"
which will train the LogisticRegressionModel by L-BFGS optimization method.

2016-06-09 7:37 GMT-07:00 chaz2505 <chaz2505@hotmail.com>:

> I ran into this problem too - it's because WeightedLeastSquares (added in
> 1.6.0 SPARK-10668) is being used on an ill-conditioned problem
> (SPARK-11918). I guess because of the one hot encoding. To get around it
> you
> need to ensure WeightedLeastSquares isn't used. Set parameters to make the
> following false:
>
> $(solver) == "auto" && $(elasticNetParam) == 0.0 &&
>       numFeatures <= WeightedLeastSquares.MAX_NUM_FEATURES) || $(solver) ==
> "normal"
>
> Hope this helps
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/Trainning-a-spark-ml-linear-regresion-model-fail-after-migrating-from-1-5-2-to-1-6-1-tp27111p27128.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Mime
View raw message