spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sonal Goyal <sonalgoy...@gmail.com>
Subject Re: LinearRegression and model prediction threshold
Date Fri, 31 Oct 2014 18:57:57 GMT
You can serialize the model to a local/hdfs file system and use it later
when you want.

Best Regards,
Sonal
Nube Technologies <http://www.nubetech.co>

<http://in.linkedin.com/in/sonalgoyal>



On Sat, Nov 1, 2014 at 12:02 AM, Sean Owen <sowen@cloudera.com> wrote:

> It sounds like you are asking about logistic regression, not linear
> regression. If so, yes that's just what it does. The default would be
> 0.5 in logistic regression. If you 'clear' the threshold you get the
> raw margin out of this and other linear classifiers.
>
> On Fri, Oct 31, 2014 at 7:18 PM, Sameer Tilak <sstilak@live.com> wrote:
> > Hi All,
> >
> > I am using LinearRegression and have a question about the details on
> > model.predict method. Basically it is predicting variable y given an
> input
> > vector x. However, can someone point me to the documentation about what
> is
> > the threshold used in the predict method? Can that be changed ? I am
> > assuming that i/p vector essentially gets mapped to a number and is
> compared
> > against a threshold value and then y is either set to 0 or 1 based on
> those
> > two numbers.
> >
> > Another question I have is if I want to save the model to hdfs for later
> > reuse is there a recommended way for doing that?
> >
> > // Building the model
> > val numIterations = 100
> > val model = LinearRegressionWithSGD.train(parsedData, numIterations)
> >
> > // Evaluate model on training examples and compute training error
> > val valuesAndPreds = parsedData.map { point =>
> >   val prediction = model.predict(point.features)
> >   (point.label, prediction)
> > }
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Mime
View raw message