mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From prasenjit mukherjee <prasen....@gmail.com>
Subject Re: problem Interpreting SVD values
Date Wed, 21 Oct 2009 02:18:15 GMT
On Tue, Oct 20, 2009 at 10:22 PM, Ted Dunning <ted.dunning@gmail.com> wrote:
> This is *exactly* the problem with LDA.  You can try putting a logistic
> regression step in the way to combine the positive or negative values into a
> [0,1] value.

Thanks for the pointer, Also  can you explain ( or refer an article )
it a little bit on how to use log regression to get a [0,1] value out
of U/V vectors.

>
> Or you could try LDA which is, essentially, a probabilistic version of SVD
> that gives you exactly what you want.

That was my first attempt. But the data is very sensitive to
overfitting/underfitting.  And since I dont  even know the approximate
L ( no. of latent vars ) it is becoming difficult for me to use
LDA/PLSI/approximate-SVD.

-Prasen

>
> On Tue, Oct 20, 2009 at 4:01 AM, prasenjit mukherjee
> <prasen.bea@gmail.com>wrote:
>
>> Thanks a bunch, I fixed the problem by using Colt.
>>
>> Also I am trying to use U/V values to assign probability p(z|u) and
>> p(z|s). My problem is how do I interpret the -ve U/V values and assign
>> a +ve probability value for that entry.
>>
>> -Prasen
>>
>> On Sun, Oct 18, 2009 at 10:58 PM, Ted Dunning <ted.dunning@gmail.com>
>> wrote:
>> > I have not worked with lingpipe, but ...
>> >
>> > When I follow the steps you are taking using R, I get this:
>> >
>> > *> docs=data.frame(d0=c(2,2,0,0), d1=c(2,2,0,0), d2=c(0,0,2,2),
>> > row.names=c("t0","t1","t2","t3"))
>> >> docs
>> >   d0 d1 d2
>> > t0  2  2  0
>> > t1  2  2  0
>> > t2  0  0  2
>> > t3  0  0  2
>> >> svd(docs)
>> > $d
>> > [1] 4.000000 2.828427 0.000000
>> >
>>
>> <trimmed/>
>>
>
>
>
> --
> Ted Dunning, CTO
> DeepDyve
>

Mime
View raw message