systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From arijit chakraborty <ak...@hotmail.com>
Subject Re: Update Spark Configuration to improve SystemML performance
Date Tue, 25 Jul 2017 09:27:52 GMT
Hi Matthias,


Thanks for your mail. I'm attaching again the server configurations. I'm also adding your
personal email id, just to be double sure you can see the images. Pardon me for that. I could
improve the setup further so that now I can run the code at the same speed as R (around 40
mins). But this setup I'm sharing is the older setup. So most probably the performance of
my code was dependent on spark configuration. So if you can help me on that.


Also, currently I'm mainly working on CNN works. And I've decent programming experience in
python & R. But I would request you to share with me project which is among the least
priority one. This will help me to get accustomed with this project setup without getting
bothered about time lines.


Thank you!

Arijit[cid:e9962ac3-ab1b-435a-b075-2f904362a093]
[cid:d8a9492b-34ab-4484-a469-f031a792f834]
[cid:346382dd-54fa-49a8-b081-897bf0074d74]

________________________________
From: Matthias Boehm <mboehm7@googlemail.com>
Sent: Tuesday, July 25, 2017 2:10:52 PM
To: dev@systemml.apache.org
Subject: Re: Update Spark Configuration to improve SystemML performance

great to hear that - we welcome additional contributions. Just let us know
in which area you're most interested in (e.g., algorithms, APIs, optimizer,
runtime, etc) and we could identifying a couple of tasks to get you started.

Regarding the performance numbers, I am not able to see the details. Also
could you share which operation was causing the large GC overhead - maybe
we can improve the runtime for the specific scenario. Thanks.

Regards,
Matthias

On Mon, Jul 24, 2017 at 12:17 PM, arijit chakraborty <akc14@hotmail.com>
wrote:

> Hi,
>
>
> I tried to work on spark configuration file to improve the systemML
> performance. Even after much tuning R code is running in 40 mins, but
> SystemML is taking 2.2 hours. Please find the spark configuration
> screenshots. Please let me know if I'm making some mistake in tuning of the
> spark configuration. One problem we could rectify is garbage time error.
> Now, it's completely not there. That was one major bottleneck which was
> making the code extremely slow.
>
>
> I'm  working in local system and created a standalone version of spark,
> with master and workers. The following are the details:
>
>
>
>
>
>
>
>
>
> I also wants to know is it possible to get involved with systemML
> development? My project is almost on the verge of completion and I learned
> a lot from you all people. And I really liked this project. So I want to
> contribute more fruitfully in it.
>
>
> Thank you!
>
> Arijit
>

Mime
View raw message