+dev list

I personally don't mind letting the regression suite run overnight. The important thing is that we do not push changes that have not passed the full automated test suite. In the interest of efficiency, we shouldn't even be reviewing most PRs until after they have passed the automated tests.

Deron, are you seeing a backlog of not-yet-started builds queueing up on the PR build server? If the queue is getting long, we can add additional machines to the Jenkins cluster.

Fred

Inactive hide details for Deron Eriksson---12/08/2016 11:06:52 AM---Hi Niketan,Deron Eriksson---12/08/2016 11:06:52 AM---Hi Niketan,

From: Deron Eriksson/San Francisco/IBM
To: Niketan Pansare/Almaden/IBM@IBMUS
Cc: Berthold Reinwald/Almaden/IBM@IBMUS, Frederick R Reiss/Almaden/IBM@IBMUS
Date: 12/08/2016 11:06 AM
Subject: Re: test suite running slowly after disable cache/sparse commit?





Hi Niketan,

Perhaps Berthold or Fred could add a little guidance here in terms of what is acceptable? Having the test suite go from 2:21 to 3:41 (one pull request yesterday took 4:11 to complete - https://sparktc.ibmcloud.com/jenkins/job/SystemML-PullRequestBuilder/909/) is very serious to me. Even if the test suite runs at 3:00, this is a serious slowdown. It slows down our ability to validate pull requests and other code on jenkins.

Deron


----- Original message -----
From: Niketan Pansare/Almaden/IBM
To: Deron Eriksson/San Francisco/IBM@ibmus
Cc: Berthold Reinwald/Almaden/IBM@ibmus, Frederick R Reiss/Almaden/IBM@ibmus
Subject: Re: test suite running slowly after disable cache/sparse commit?
Date: Thu, Dec 8, 2016 8:55 AM

Hi Deron,

The commit replicated application tests for disable sparse and disable caching. So, the test time should increase. We should increase the duration or reduce the number of application tests we want to test with caching and sparse disabled.

Thanks

Niketan

On Dec 8, 2016, at 7:47 AM, Deron Eriksson <
deron@us.ibm.com> wrote: