mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Hans" <>
Subject Re: Looking for Linear Regression on Hadoop
Date Tue, 08 Dec 2009 08:14:50 GMT
Hey Rajat,

> After perusing the docs on the Mahout site, it seems like the
> following algorithms havent been implemented yet-
> Locally-Weighted Linear Regression
> Linear Regression

Implementing LWLR was an initial goal of the project since LWLR is also
mentioned in the Stanford paper that talks about doing machine learning in
a map-reduce way. I said I would look into implementing it a long time ago
(maybe a year or even one and a half) but so far just haven't gotten
around to actually do it. I don't think that it would be too much work,
maybe a weekend and some evening. I probably just should try to get my
shit together and just implement it. Now there would be a bit more
motivation with knowing that there's someone who would actually use it.

Linear Regression is just a degenerated LWLR where all weights are equal
to 1.

> Basically, there is a stock market phenomenon which I'm trying to
> predict. It is called a short squeeze. I have about 16,000 data points
> - stocks and a point in time where the phenomenon has occurred. I'm
> trying to develop a predictive model in a hadoop cluster.

As others have already pointed out, you wouldn't see a noticable
difference when using Mahout to do this. It could easily be done on a
single machine. However, if it's not about this particular problem but
about a principle implementation and showing that a speedup is possible,
it would make sense to implement it using Mahout/Hadoop. But for just
solving the regression problem I would just code it in Matlab (one-liner
using the \ operator).


PGP Public Key:
Fingerprint: E110 4CA3 288A 93F3 5237  E904 A85B 4B18 CFDC 63E3

View raw message