spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dongjoon Hyun <dongjoon.h...@gmail.com>
Subject Re: Migration `Spark QA Compile` Jenkins jobs to GitHub Action
Date Tue, 19 Nov 2019 22:29:36 GMT
Thank you, Sean, Shane, and Xiao!

Bests,
Dongjoon.

On Tue, Nov 19, 2019 at 2:15 PM Shane Knapp <sknapp@berkeley.edu> wrote:

> i had a few minutes aaaaand everything has been deleted!
>
> On Tue, Nov 19, 2019 at 2:02 PM Shane Knapp <sknapp@berkeley.edu> wrote:
> >
> > thank sean!
> >
> > i am all for moving these jobs to github actions, and will be doing
> > this 'soon' as i'm @ kubecon this week.
> >
> > btw the R ecosystem definitely needs some attention, however, but
> > that's an issue for another time.  :)
> >
> > On Tue, Nov 19, 2019 at 1:49 PM Sean Owen <srowen@gmail.com> wrote:
> > >
> > > I would favor moving whatever we can to Github. It's difficult to
> > > modify the Jenkins instances without Shane's valiant help, and over
> > > time makes more sense to modernize and integrate it into the project.
> > >
> > > On Tue, Nov 19, 2019 at 3:35 PM Dongjoon Hyun <dongjoon.hyun@gmail.com>
> wrote:
> > > >
> > > > Hi, All.
> > > >
> > > > Apache Spark community used the following dashboard as post-hook
> verifications.
> > > >
> > > >
> https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Compile/
> > > >
> > > > There are six registered jobs.
> > > >
> > > >     1. spark-branch-2.4-compile-maven-hadoop-2.6
> > > >     2. spark-branch-2.4-compile-maven-hadoop-2.7
> > > >     3. spark-branch-2.4-lint
> > > >     4. spark-master-compile-maven-hadoop-2.7
> > > >     5. spark-master-compile-maven-hadoop-3.2
> > > >     6. spark-master-lint
> > > >
> > > > Now, we added `GitHub Action` jobs. You can see the green check at
> every commit.
> > > >
> > > >     https://github.com/apache/spark/commits/master
> > > >     https://github.com/apache/spark/commits/branch-2.4
> > > >
> > > > If you click the green check, you can see the detail.
> > > > The followings are the example runs at the last commits on both
> branches.
> > > >
> > > >     https://github.com/apache/spark/runs/310411948 (master)
> > > >     https://github.com/apache/spark/runs/309522646 (branch-2.4)
> > > >
> > > > New `GitHub Action` have more combination than the old Jenkins jobs.
> > > >
> > > >     - branch-2.4-scala-2.11-hadoop-2.6 (compile/package/install)
> > > >     - branch-2.4-scala-2.12-hadoop-2.6 (compile/package/install)
> > > >     - branch-2.4-scala-2.11-hadoop-2.7 (compile/package/install)
> > > >     - branch-2.4-scala-2.12-hadoop-2.7 (compile/package/install)
> > > >     - branch-2.4-linters (Scala/Java/Python/R)
> > > >     - master-scala-2.12-hadoop-2.7 (compile/package/install)
> > > >     - master-scala-2.12-hadoop-3.2 (compile/package/install)
> > > >     - master-scala-2.12-hadoop-3.2-jdk11 (compile/package/install)
> > > >     - master-linters (Scala/Java/Python/R)
> > > >
> > > > In addition, this is a part of Apache Spark code base and everyone
> can make contributions on this.
> > > >
> > > > Finally, as the last piece of this work, we are going to remove the
> legacy Jenkins jobs via the following JIRA issue.
> > > >
> > > >     https://issues.apache.org/jira/browse/SPARK-29935
> > > >
> > > > Please let me know if you have any concerns on this.
> > > > (We can keep the legacy jobs, but two of them are already broken.)
> > > >
> > > > Bests,
> > > > Dongjoon.
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
> > >
> >
> >
> > --
> > Shane Knapp
> > UC Berkeley EECS Research / RISELab Staff Technical Lead
> > https://rise.cs.berkeley.edu
>
>
>
> --
> Shane Knapp
> UC Berkeley EECS Research / RISELab Staff Technical Lead
> https://rise.cs.berkeley.edu
>

Mime
View raw message