mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Osman Başkaya <osman.bask...@computer.org>
Subject Re: Changing in-memory DataModel to a DB dependent only DataModel after building recommender
Date Mon, 21 Jan 2013 20:57:18 GMT
Hey Sean,

Once you compute user and item factors, you don't need to create a dense
matrix in which all the user's predicted ratings contains. Of course, one
dependency remains: you may want to know which user rates which movies.
Otherwise you recommend movies which are rated already. However, computing
all predictions and storing them does not provides this, neither.

On Mon, Jan 21, 2013 at 10:40 PM, Sean Owen <srowen@gmail.com> wrote:

> If you don't have the data in memory you can't compute anything. The
> recommender itself doesn't do anything without data. That's why it
> seemed like you really just wanted to compute everything offline
> first, in which case the simplest solution is to store it however you
> like and fetch that result however you like.
>
> On Mon, Jan 21, 2013 at 8:22 PM, Ceyhun Can ÜLKER <ceyhuncanu@gmail.com>
> wrote:
> > Hi again,
> >
> > Thank you for your quick reply, Sean. I couldn't understand one point.
> What
> > do you mean by pre-compute and store recommendations? Doesn't it mean
> > having a dense (rather filled?) rating matrix? So it would make memory
> > usage much worse, even if it is possible. Wouldn't it better to keep the
> > model and compute whenever necessary?
> >
> > Thanks
> > Ceyhun Can Ulker
> >
> >
> > On Mon, Jan 21, 2013 at 9:58 PM, Sean Owen <srowen@gmail.com> wrote:
> >
> >> You would have to write this yourself, yes.
> >> If you're not keeping the data in memory, you're not updating the
> >> results in real-time. So there's no real need to keep any DataModel
> >> around at all. Just pre-compute and store recommendations and update
> >> them periodically. Nothing has to be on-line then.
> >>
> >> On Mon, Jan 21, 2013 at 7:54 PM, Ceyhun Can ÜLKER <ceyhuncanu@gmail.com
> >
> >> wrote:
> >> > Hello,
> >> >
> >> > In our application we are using ReloadFromJDBCDataModel for its speed
> >> > advantage of in-memory representation and being able to update
> >> periodically
> >> > to pull in new data from a database source.
> >> >
> >> > However, once the recommender is build we do not want to keep the
> ratings
> >> > data in memory (we would like to query the database when rating data
> is
> >> > needed). We want to replace the ReloadFromJDBCDataModel with a
> >> > MySqlJDBCDataModel after build. But there is no setter method for it,
> >> > furthermore, the field that keeps the DataModel is in
> AbstractRecommender
> >> > (superclass of SVDRecommender) and it is declared final.
> >> >
> >> > We thought we could write a new class that derives from DataModel,
> which
> >> > initial keeps a Reload model instance (let's call this delegateModel),
> >> has
> >> > a setter method for it, and delegates all DataModel methods, so that
> we
> >> > could set this delegateModel field to another instance, say
> >> > MySqlJDBCDataModel instance. Is this a good method for removing
> in-memory
> >> > representation dependency after the build process?
> >> >
> >> > How can we achieve this change? Or is there an alternative and better
> way
> >> > to achieve this?
> >> >
> >> > Thanks
> >> > Ceyhun Can Ulker
> >>
>



-- 
Osman Başkaya
Koc University
MS Student | Computer Science and Engineering

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message