spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Irving Duran <irving.du...@gmail.com>
Subject Re: ML Linear and Logistic Regression - Poor Performance
Date Fri, 27 Apr 2018 19:50:14 GMT
Are you reformatting the data correctly for logistic (meaning 0 & 1's)
before modeling?  What are OS and spark version you using?

Thank You,

Irving Duran


On Fri, Apr 27, 2018 at 2:34 PM Thodoris Zois <zois@ics.forth.gr> wrote:

> Hello,
>
> I am running an experiment to test logistic and linear regression on spark
> using MLlib.
>
> My dataset is only 128MB and something weird happens. Linear regression
> takes about 127 seconds either with 1 or 500 iterations. On the other hand,
> logistic regression most of the times does not manage to finish either with
> 1 iteration. I usually get memory heap error.
>
> In both cases I use the default cores and memory for driver and I spawn 1
> executor with 1 core and 2GBs of memory.
>
> Except that, I get a warning about NativeBLAS. I searched in the Internet
> and I found that I have to install libgfortran. Even if I did it the
> warning remains.
>
> Any ideas for the above?
>
> Thank you,
> - Thodoris
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: user-unsubscribe@spark.apache.org
>
>

Mime
View raw message