spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Owen <sro...@gmail.com>
Subject Re: Migration `Spark QA Compile` Jenkins jobs to GitHub Action
Date Tue, 19 Nov 2019 21:49:29 GMT
I would favor moving whatever we can to Github. It's difficult to
modify the Jenkins instances without Shane's valiant help, and over
time makes more sense to modernize and integrate it into the project.

On Tue, Nov 19, 2019 at 3:35 PM Dongjoon Hyun <dongjoon.hyun@gmail.com> wrote:
>
> Hi, All.
>
> Apache Spark community used the following dashboard as post-hook verifications.
>
>     https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Compile/
>
> There are six registered jobs.
>
>     1. spark-branch-2.4-compile-maven-hadoop-2.6
>     2. spark-branch-2.4-compile-maven-hadoop-2.7
>     3. spark-branch-2.4-lint
>     4. spark-master-compile-maven-hadoop-2.7
>     5. spark-master-compile-maven-hadoop-3.2
>     6. spark-master-lint
>
> Now, we added `GitHub Action` jobs. You can see the green check at every commit.
>
>     https://github.com/apache/spark/commits/master
>     https://github.com/apache/spark/commits/branch-2.4
>
> If you click the green check, you can see the detail.
> The followings are the example runs at the last commits on both branches.
>
>     https://github.com/apache/spark/runs/310411948 (master)
>     https://github.com/apache/spark/runs/309522646 (branch-2.4)
>
> New `GitHub Action` have more combination than the old Jenkins jobs.
>
>     - branch-2.4-scala-2.11-hadoop-2.6 (compile/package/install)
>     - branch-2.4-scala-2.12-hadoop-2.6 (compile/package/install)
>     - branch-2.4-scala-2.11-hadoop-2.7 (compile/package/install)
>     - branch-2.4-scala-2.12-hadoop-2.7 (compile/package/install)
>     - branch-2.4-linters (Scala/Java/Python/R)
>     - master-scala-2.12-hadoop-2.7 (compile/package/install)
>     - master-scala-2.12-hadoop-3.2 (compile/package/install)
>     - master-scala-2.12-hadoop-3.2-jdk11 (compile/package/install)
>     - master-linters (Scala/Java/Python/R)
>
> In addition, this is a part of Apache Spark code base and everyone can make contributions
on this.
>
> Finally, as the last piece of this work, we are going to remove the legacy Jenkins jobs
via the following JIRA issue.
>
>     https://issues.apache.org/jira/browse/SPARK-29935
>
> Please let me know if you have any concerns on this.
> (We can keep the legacy jobs, but two of them are already broken.)
>
> Bests,
> Dongjoon.

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org


Mime
View raw message