mahout-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Lyubimov <dlie...@gmail.com>
Subject Re: SSVD Parameters Constrains
Date Sun, 07 Oct 2012 22:02:21 GMT
PS. Also keep in mind that if you use multiple files as input, and
their sizes are smaller than hdfs split size, it will also mean that
some splits will have reduced size even if the total input size looks
benign. I think there was at least one case (which also pertains to
"problem too smail" case) where a user discovered that one of the
input files of distributed row matrix had less than k+p rows hence
setting off this block height deficiency problem.

-d

On Sun, Oct 7, 2012 at 2:55 PM, Dmitriy Lyubimov <dlieu.7@gmail.com> wrote:
> Ahmed, if you are getting this, in all cases people talked about it it
> meant their problem was too small.  If A has m x n geometry, then it
> must be true that k+p<=min(m,n).
>
> Another possible reason is if height of blocks of A crerated in the
> mappers are less than k+p. In practice we yet to see a problem that
> actually may ever run into condition (although it is definitely
> possible if you occasionally have very dense very large row vectors so
> they take up enough space to create split block height problem). If it
> is indeed split block height problem, then the remedy is to increase
> split size either by hadoop parameter or (i think) one of the SSVD
> command line parameters. Although like i said nobody yet ran into
> block height deficiency problem yet so i have no knowledge of verified
> resolution of this problem by means of manipulating hadoop parameter
> setup in Mahout.
>
> -d
>
> On Sun, Oct 7, 2012 at 2:16 PM, Ahmed Elgohary <aagohary@gmail.com> wrote:
>> Hi,
>>
>> Can someone list all the constrains on the parameters (k,p &aBlockRows)
>> that should be satisfied in order for the Q-job in ssvd to work fine? I
>> tried many values, made sure that (k+p<=m & k+p<=n & p is in the range
20
>> .. 200). but I am still getting the errors: "Givens thin QR: must be true:
>> m>=n" or ""new m can't be less than n".
>>
>> thanks,
>>
>> --ahmed

Mime
View raw message