spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dongjoon Hyun <dongjoon.h...@gmail.com>
Subject Migration `Spark QA Compile` Jenkins jobs to GitHub Action
Date Tue, 19 Nov 2019 21:35:39 GMT
Hi, All.

Apache Spark community used the following dashboard as post-hook
verifications.

    https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Compile/

There are six registered jobs.

    1. spark-branch-2.4-compile-maven-hadoop-2.6
    2. spark-branch-2.4-compile-maven-hadoop-2.7
    3. spark-branch-2.4-lint
    4. spark-master-compile-maven-hadoop-2.7
    5. spark-master-compile-maven-hadoop-3.2
    6. spark-master-lint

Now, we added `GitHub Action` jobs. You can see the green check at every
commit.

    https://github.com/apache/spark/commits/master
    https://github.com/apache/spark/commits/branch-2.4

If you click the green check, you can see the detail.
The followings are the example runs at the last commits on both branches.

    https://github.com/apache/spark/runs/310411948 (master)
    https://github.com/apache/spark/runs/309522646 (branch-2.4)

New `GitHub Action` have more combination than the old Jenkins jobs.

    - branch-2.4-scala-2.11-hadoop-2.6 (compile/package/install)
    - branch-2.4-scala-2.12-hadoop-2.6 (compile/package/install)
    - branch-2.4-scala-2.11-hadoop-2.7 (compile/package/install)
    - branch-2.4-scala-2.12-hadoop-2.7 (compile/package/install)
    - branch-2.4-linters (Scala/Java/Python/R)
    - master-scala-2.12-hadoop-2.7 (compile/package/install)
    - master-scala-2.12-hadoop-3.2 (compile/package/install)
    - master-scala-2.12-hadoop-3.2-jdk11 (compile/package/install)
    - master-linters (Scala/Java/Python/R)

In addition, this is a part of Apache Spark code base and everyone can make
contributions on this.

Finally, as the last piece of this work, we are going to remove the legacy
Jenkins jobs via the following JIRA issue.

    https://issues.apache.org/jira/browse/SPARK-29935

Please let me know if you have any concerns on this.
(We can keep the legacy jobs, but two of them are already broken.)

Bests,
Dongjoon.

Mime
View raw message