spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thodoris Zois <>
Subject ML Linear and Logistic Regression - Poor Performance
Date Fri, 27 Apr 2018 19:34:17 GMT

I am running an experiment to test logistic and linear regression on spark using MLlib.

My dataset is only 128MB and something weird happens. Linear regression takes about 127 seconds
either with 1 or 500 iterations. On the other hand, logistic regression most of the times
does not manage to finish either with 1 iteration. I usually get memory heap error.

In both cases I use the default cores and memory for driver and I spawn 1 executor with 1
core and 2GBs of memory. 

Except that, I get a warning about NativeBLAS. I searched in the Internet and I found that
I have to install libgfortran. Even if I did it the warning remains.

Any ideas for the above?

Thank you,
- Thodoris

To unsubscribe e-mail:

View raw message