spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Owen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-3431) Parallelize execution of tests
Date Tue, 09 Dec 2014 18:57:13 GMT

     [ https://issues.apache.org/jira/browse/SPARK-3431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sean Owen updated SPARK-3431:
-----------------------------
    Attachment: SPARK-3431-srowen-attempt.patch

FWIW I tried this as well from scratch, and it all *looks* pretty OK except that a bunch of
tests fail straight away. I kind of don't know what to make of it, since you would expect
errors more of the 'no tests found' variety or something to do with the harness. I see the
same serialization errors. Maybe something ends up being different about the scalatest classpath.

So, I'm pausing on this as using surefire is a nice-to-have but not necessarily essential
to parallelization. I'm looking at the sort of related http://issues.apache.org/jira/browse/SPARK-4159
which is about actually running Java tests in scalatest. Maybe I'll learn something.

> Parallelize execution of tests
> ------------------------------
>
>                 Key: SPARK-3431
>                 URL: https://issues.apache.org/jira/browse/SPARK-3431
>             Project: Spark
>          Issue Type: Improvement
>          Components: Build
>            Reporter: Nicholas Chammas
>            Assignee: Nicholas Chammas
>         Attachments: SPARK-3431-srowen-attempt.patch
>
>
> Running all the tests in {{dev/run-tests}} takes up to 2 hours. A common strategy to
cut test time down is to parallelize the execution of the tests. Doing that may in turn require
some prerequisite changes to be made to how certain tests run.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message