On Thu, Mar 12, 2015 at 3:05 PM, Jaonary Rabarisoa <jaonary@gmail.com>
wrote:
> In fact, by activating netlib with native libraries it goes faster.
>
> Glad you got it work ! Better performance was one of the reasons we made
the switch.
> Thanks
>
> On Tue, Mar 10, 2015 at 7:03 PM, Shivaram Venkataraman <
> shivaram@eecs.berkeley.edu> wrote:
>
>> There are a couple of differences between the mlmatrix implementation
>> and the one used in AMPCamp
>>
>>  I think the AMPCamp one uses JBLAS which tends to ship native BLAS
>> libraries along with it. In mlmatrix we switched to using Breeze + Netlib
>> BLAS which is faster but needs some setup [1] to pick up native libraries.
>> If native libraries are not found it falls back to a JVM implementation, so
>> that might explain the slow down.
>>
>>  The other difference if you are comparing the whole image pipeline is
>> that I think the AMPCamp version used NormalEquations which is around 23x
>> faster (just in terms of number of flops) compared to TSQR.
>>
>> [1]
>> https://github.com/fommil/netlibjava#machineoptimisedsystemlibraries
>>
>> Thanks
>> Shivaram
>>
>> On Tue, Mar 10, 2015 at 9:57 AM, Jaonary Rabarisoa <jaonary@gmail.com>
>> wrote:
>>
>>> I'm trying to play with the implementation of least square solver (Ax =
>>> b) in mlmatrix.TSQR where A is a 50000*1024 matrix and b a 50000*10
>>> matrix. It works but I notice
>>> that it's 8 times slower than the implementation given in the latest
>>> ampcamp :
>>> http://ampcamp.berkeley.edu/5/exercises/imageclassificationwithpipelines.html
>>> . As far as I know these two implementations come from the same basis.
>>> What is the difference between these two codes ?
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Mar 3, 2015 at 8:02 PM, Shivaram Venkataraman <
>>> shivaram@eecs.berkeley.edu> wrote:
>>>
>>>> There are couple of solvers that I've written that is part of the
>>>> AMPLab mlmatrix repo [1,2]. These aren't part of MLLib yet though and if
>>>> you are interested in porting them I'd be happy to review it
>>>>
>>>> Thanks
>>>> Shivaram
>>>>
>>>>
>>>> [1]
>>>> https://github.com/amplab/mlmatrix/blob/master/src/main/scala/edu/berkeley/cs/amplab/mlmatrix/TSQR.scala
>>>> [2]
>>>> https://github.com/amplab/mlmatrix/blob/master/src/main/scala/edu/berkeley/cs/amplab/mlmatrix/NormalEquations.scala
>>>>
>>>> On Tue, Mar 3, 2015 at 9:01 AM, Jaonary Rabarisoa <jaonary@gmail.com>
>>>> wrote:
>>>>
>>>>> Dear all,
>>>>>
>>>>> Is there a least square solver based on DistributedMatrix that we can
>>>>> use out of the box in the current (or the master) version of spark ?
>>>>> It seems that the only least square solver available in spark is
>>>>> private to recommender package.
>>>>>
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Jao
>>>>>
>>>>
>>>>
>>>
>>
>
