spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shane Knapp <skn...@berkeley.edu>
Subject Re: Migration `Spark QA Compile` Jenkins jobs to GitHub Action
Date Tue, 19 Nov 2019 22:02:01 GMT
thank sean!

i am all for moving these jobs to github actions, and will be doing
this 'soon' as i'm @ kubecon this week.

btw the R ecosystem definitely needs some attention, however, but
that's an issue for another time.  :)

On Tue, Nov 19, 2019 at 1:49 PM Sean Owen <srowen@gmail.com> wrote:
>
> I would favor moving whatever we can to Github. It's difficult to
> modify the Jenkins instances without Shane's valiant help, and over
> time makes more sense to modernize and integrate it into the project.
>
> On Tue, Nov 19, 2019 at 3:35 PM Dongjoon Hyun <dongjoon.hyun@gmail.com> wrote:
> >
> > Hi, All.
> >
> > Apache Spark community used the following dashboard as post-hook verifications.
> >
> >     https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Compile/
> >
> > There are six registered jobs.
> >
> >     1. spark-branch-2.4-compile-maven-hadoop-2.6
> >     2. spark-branch-2.4-compile-maven-hadoop-2.7
> >     3. spark-branch-2.4-lint
> >     4. spark-master-compile-maven-hadoop-2.7
> >     5. spark-master-compile-maven-hadoop-3.2
> >     6. spark-master-lint
> >
> > Now, we added `GitHub Action` jobs. You can see the green check at every commit.
> >
> >     https://github.com/apache/spark/commits/master
> >     https://github.com/apache/spark/commits/branch-2.4
> >
> > If you click the green check, you can see the detail.
> > The followings are the example runs at the last commits on both branches.
> >
> >     https://github.com/apache/spark/runs/310411948 (master)
> >     https://github.com/apache/spark/runs/309522646 (branch-2.4)
> >
> > New `GitHub Action` have more combination than the old Jenkins jobs.
> >
> >     - branch-2.4-scala-2.11-hadoop-2.6 (compile/package/install)
> >     - branch-2.4-scala-2.12-hadoop-2.6 (compile/package/install)
> >     - branch-2.4-scala-2.11-hadoop-2.7 (compile/package/install)
> >     - branch-2.4-scala-2.12-hadoop-2.7 (compile/package/install)
> >     - branch-2.4-linters (Scala/Java/Python/R)
> >     - master-scala-2.12-hadoop-2.7 (compile/package/install)
> >     - master-scala-2.12-hadoop-3.2 (compile/package/install)
> >     - master-scala-2.12-hadoop-3.2-jdk11 (compile/package/install)
> >     - master-linters (Scala/Java/Python/R)
> >
> > In addition, this is a part of Apache Spark code base and everyone can make contributions
on this.
> >
> > Finally, as the last piece of this work, we are going to remove the legacy Jenkins
jobs via the following JIRA issue.
> >
> >     https://issues.apache.org/jira/browse/SPARK-29935
> >
> > Please let me know if you have any concerns on this.
> > (We can keep the legacy jobs, but two of them are already broken.)
> >
> > Bests,
> > Dongjoon.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>


-- 
Shane Knapp
UC Berkeley EECS Research / RISELab Staff Technical Lead
https://rise.cs.berkeley.edu

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org


Mime
View raw message