systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthias Boehm <mboe...@googlemail.com>
Subject Re: Update Spark Configuration to improve SystemML performance
Date Tue, 25 Jul 2017 08:40:52 GMT
great to hear that - we welcome additional contributions. Just let us know
in which area you're most interested in (e.g., algorithms, APIs, optimizer,
runtime, etc) and we could identifying a couple of tasks to get you started.

Regarding the performance numbers, I am not able to see the details. Also
could you share which operation was causing the large GC overhead - maybe
we can improve the runtime for the specific scenario. Thanks.

Regards,
Matthias

On Mon, Jul 24, 2017 at 12:17 PM, arijit chakraborty <akc14@hotmail.com>
wrote:

> Hi,
>
>
> I tried to work on spark configuration file to improve the systemML
> performance. Even after much tuning R code is running in 40 mins, but
> SystemML is taking 2.2 hours. Please find the spark configuration
> screenshots. Please let me know if I'm making some mistake in tuning of the
> spark configuration. One problem we could rectify is garbage time error.
> Now, it's completely not there. That was one major bottleneck which was
> making the code extremely slow.
>
>
> I'm  working in local system and created a standalone version of spark,
> with master and workers. The following are the details:
>
>
>
>
>
>
>
>
>
> I also wants to know is it possible to get involved with systemML
> development? My project is almost on the verge of completion and I learned
> a lot from you all people. And I really liked this project. So I want to
> contribute more fruitfully in it.
>
>
> Thank you!
>
> Arijit
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message