spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Masood Krohy <masood.kr...@intact.net>
Subject Re: LinearRegressionWithSGD and Rank Features By Importance
Date Mon, 07 Nov 2016 15:31:03 GMT
If you go down this route (look at actual coefficients/weights), then make 
sure your features are scaled first and have more or less the same mean 
when feeding them into the algo. If not, then actual coefficients/weights 
wouldn't tell you much. In any case, SGD performs badly with unscaled 
features, so you gain if you scale the features beforehand.
Masood

------------------------------
Masood Krohy, Ph.D.
Data Scientist, Intact Lab-R&D
Intact Financial Corporation
http://ca.linkedin.com/in/masoodkh



De :    Carlo.Allocca <carlo.allocca@open.ac.uk>
A :     Mohit Jaggi <mohitjaggi@gmail.com>
Cc :    Carlo.Allocca <carlo.allocca@open.ac.uk>, "user@spark.apache.org" 
<user@spark.apache.org>
Date :  2016-11-04 03:39
Objet : Re: LinearRegressionWithSGD and Rank Features By Importance



Hi Mohit, 

Thank you for your reply. 
OK. it means coefficient with high score are more important that other 
with low score…

Many Thanks,
Best Regards,
Carlo


> On 3 Nov 2016, at 20:41, Mohit Jaggi <mohitjaggi@gmail.com> wrote:
> 
> For linear regression, it should be fairly easy. Just sort the 
co-efficients :)
> 
> Mohit Jaggi
> Founder,
> Data Orchard LLC
> www.dataorchardllc.com
> 
> 
> 
> 
>> On Nov 3, 2016, at 3:35 AM, Carlo.Allocca <carlo.allocca@open.ac.uk> 
wrote:
>> 
>> Hi All,
>> 
>> I am using SPARK and in particular the MLib library.
>> 
>> import org.apache.spark.mllib.regression.LabeledPoint;
>> import org.apache.spark.mllib.regression.LinearRegressionModel;
>> import org.apache.spark.mllib.regression.LinearRegressionWithSGD;
>> 
>> For my problem I am using the LinearRegressionWithSGD and I would like 
to perform a “Rank Features By Importance”.
>> 
>> I checked the documentation and it seems that does not provide such 
methods.
>> 
>> Am I missing anything?  Please, could you provide any help on this?
>> Should I change the approach?
>> 
>> Many Thanks in advance,
>> 
>> Best Regards,
>> Carlo
>> 
>> 
>> -- The Open University is incorporated by Royal Charter (RC 000391), an 
exempt charity in England & Wales and a charity registered in Scotland (SC 
038302). The Open University is authorised and regulated by the Financial 
Conduct Authority.
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>> 
> 


---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org




Mime
View raw message