spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicholas Chammas <nicholas.cham...@gmail.com>
Subject Re: Unit tests in < 5 minutes
Date Sat, 06 Dec 2014 18:54:29 GMT
Ted,

I posted some updates
<https://issues.apache.org/jira/browse/SPARK-3431?focusedCommentId=14236540&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14236540>
on
JIRA on my progress (or lack thereof) getting SBT to parallelize test
suites properly. I'm currently stuck with SBT / ScalaTest, so I may move on
to trying Maven.

Andrew,

Once we have a basic grasp of how to parallelize some of the tests, the
next step will probably be to use containers (i.e. Docker) to allow more
parallelization, especially for those tests that, for example, contend for
ports.

Nick

On Fri Dec 05 2014 at 2:05:29 PM Andrew Or <andrew@databricks.com> wrote:

> @Patrick and Josh actually we went even further than that. We simply
> disable the UI for most tests and these used to be the single largest
> source of port conflict.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message