spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lihu <>
Subject how to speed the count operation
Date Wed, 14 May 2014 05:25:00 GMT
    I used the MLlib of spark to run some experiments, such as the lasso,
and linear regression etc. I just use the given model in MLlib:
LassoWithSGD, LinearRegressionWithSGD. but I found that the count operation
is very slow , just as show below, it seems get stuck.

   My spark version is 0.9, memory of each node is 80G. the size of dataset
is about 8G., and I run in 30 nodes.

   Any suggestion is thankful .

View raw message