systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From arijit chakraborty <ak...@hotmail.com>
Subject Update Spark Configuration to improve SystemML performance
Date Mon, 24 Jul 2017 19:17:04 GMT
Hi,


I tried to work on spark configuration file to improve the systemML performance. Even after
much tuning R code is running in 40 mins, but SystemML is taking 2.2 hours. Please find the
spark configuration screenshots. Please let me know if I'm making some mistake in tuning of
the spark configuration. One problem we could rectify is garbage time error. Now, it's completely
not there. That was one major bottleneck which was making the code extremely slow.


I'm  working in local system and created a standalone version of spark, with master and workers.
The following are the details:


[cid:848caeff-0726-4375-94ea-4874fd1b5f29]



[cid:e03403d6-c902-45a9-8a40-4c2687a1a527]


[cid:4e754a3d-78b1-4462-b8a1-eba7ec403d66]


I also wants to know is it possible to get involved with systemML development? My project
is almost on the verge of completion and I learned a lot from you all people. And I really
liked this project. So I want to contribute more fruitfully in it.


Thank you!

Arijit

Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message