Yes, WeightedLeastSquares can not solve some ill-conditioned problem currently, the community members have paid some efforts to resolve it (SPARK-13777). For the work around, you can set the solver to "l-bfgs" which will train the LogisticRegressionModel by L-BFGS optimization method.

2016-06-09 7:37 GMT-07:00 chaz2505 <>:
I ran into this problem too - it's because WeightedLeastSquares (added in
1.6.0 SPARK-10668) is being used on an ill-conditioned problem
(SPARK-11918). I guess because of the one hot encoding. To get around it you
need to ensure WeightedLeastSquares isn't used. Set parameters to make the
following false:

$(solver) == "auto" && $(elasticNetParam) == 0.0 &&
      numFeatures <= WeightedLeastSquares.MAX_NUM_FEATURES) || $(solver) ==

Hope this helps

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail: