spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Naftali Harris <naft...@affirm.com>
Subject No Intercept for Python
Date Mon, 16 Jun 2014 19:19:16 GMT
Hi everyone,

The Python LogisticRegressionWithSGD does not appear to estimate an
intercept.  When I run the following, the returned weights and intercept
are both 0.0:

from pyspark import SparkContext
from pyspark.mllib.regression import LabeledPoint
from pyspark.mllib.classification import LogisticRegressionWithSGD

def main():
    sc = SparkContext(appName="NoIntercept")

    train = sc.parallelize([LabeledPoint(0, [0]), LabeledPoint(1, [0]),
LabeledPoint(1, [0])])

    model = LogisticRegressionWithSGD.train(train, iterations=500, step=0.1)
    print "Final weights: " + str(model.weights)
    print "Final intercept: " + str(model.intercept)

if __name__ == "__main__":
    main()


Of course, one can fit an intercept with the simple expedient of adding a
column of ones, but that's kind of annoying.  Moreover, it looks like the
scala version has an intercept option.

Am I missing something? Should I just add the column of ones? If I
submitted a PR doing that, is that the sort of thing you guys would accept?

Thanks! :-)

Naftali

Mime
View raw message