spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reynold Xin <r...@databricks.com>
Subject Re: Running lint-java during PR builds?
Date Mon, 21 May 2018 18:18:15 GMT
Can we look into if there is a plugin for sbt that works and then we can
put everything into one single builder?

On Mon, May 21, 2018 at 11:17 AM Dongjoon Hyun <dongjoon.hyun@gmail.com>
wrote:

> Thank you for reconsidering this, Hyukjin. :)
>
> Bests,
> Dongjoon.
>
>
> On Mon, May 21, 2018 at 9:20 AM, Marcelo Vanzin <vanzin@cloudera.com>
> wrote:
>
>> Is there a way to trigger it conditionally? e.g. only if the diff
>> touches java files.
>>
>> On Mon, May 21, 2018 at 9:17 AM, Felix Cheung <felixcheung_m@hotmail.com>
>> wrote:
>> > One concern is with the volume of test runs on Travis.
>> >
>> > In ASF projects Travis could get significantly
>> > backed up since - if I recall - all of ASF shares one queue.
>> >
>> > At the number of PRs Spark has this could be a big issue.
>> >
>> >
>> > ________________________________
>> > From: Marcelo Vanzin <vanzin@cloudera.com>
>> > Sent: Monday, May 21, 2018 9:08:28 AM
>> > To: Hyukjin Kwon
>> > Cc: Dongjoon Hyun; dev
>> > Subject: Re: Running lint-java during PR builds?
>> >
>> > I'm fine with it. I tried to use the existing checkstyle sbt plugin
>> > (trying to fix SPARK-22269), but it depends on an ancient version of
>> > checkstyle, and I don't know sbt enough to figure out how to hack
>> > classpaths and class loaders when applying rules, so gave up.
>> >
>> > On Mon, May 21, 2018 at 1:47 AM, Hyukjin Kwon <gurwls223@gmail.com>
>> wrote:
>> >> I am going to open an INFRA JIRA if there's no explicit objection in
>> few
>> >> days.
>> >>
>> >> 2018-05-21 13:09 GMT+08:00 Hyukjin Kwon <gurwls223@gmail.com>:
>> >>>
>> >>> I would like to revive this proposal. Travis CI. Shall we give this
>> try?
>> >>> I
>> >>> think it's worth trying it.
>> >>>
>> >>> 2016-11-17 3:50 GMT+08:00 Dongjoon Hyun <dongjoon@apache.org>:
>> >>>>
>> >>>> Hi, Marcelo and Ryan.
>> >>>>
>> >>>> That was the main purpose of my proposal about Travis.CI.
>> >>>> IMO, that is the only way to achieve that without any harmful
>> >>>> side-effect
>> >>>> on Jenkins infra.
>> >>>>
>> >>>> Spark is already ready for that. Like AppVoyer, if one of you files
>> an
>> >>>> INFRA jira issue to enable that, they will turn on that. Then, we
can
>> >>>> try it
>> >>>> and see the result. Also, you can turn off easily again if you don't
>> >>>> want.
>> >>>>
>> >>>> Without this, we will consume more community efforts. For example,
we
>> >>>> merged lint-java error fix PR seven hours ago, but the master branch
>> >>>> still
>> >>>> has one lint-java error.
>> >>>>
>> >>>> https://travis-ci.org/dongjoon-hyun/spark/jobs/176351319
>> >>>>
>> >>>> Actually, I've been monitoring the history here. (It's synced every
>> 30
>> >>>> minutes.)
>> >>>>
>> >>>> https://travis-ci.org/dongjoon-hyun/spark/builds
>> >>>>
>> >>>> Could we give a change to this?
>> >>>>
>> >>>> Bests,
>> >>>> Dongjoon.
>> >>>>
>> >>>> On 2016-11-15 13:40 (-0800), "Shixiong(Ryan) Zhu"
>> >>>> <shixiong@databricks.com> wrote:
>> >>>> > I remember it's because you need to run `mvn install` before
>> running
>> >>>> > lint-java if the maven cache is empty, and `mvn install` is
pretty
>> >>>> > heavy.
>> >>>> >
>> >>>> > On Tue, Nov 15, 2016 at 1:21 PM, Marcelo Vanzin <
>> vanzin@cloudera.com>
>> >>>> > wrote:
>> >>>> >
>> >>>> > > Hey all,
>> >>>> > >
>> >>>> > > Is there a reason why lint-java is not run during PR builds?
I
>> see
>> >>>> > > it
>> >>>> > > seems to be maven-only, is it really expensive to run
after an
>> sbt
>> >>>> > > build?
>> >>>> > >
>> >>>> > > I see a lot of PRs coming in to fix Java style issues,
and those
>> all
>> >>>> > > seem a little unnecessary. Either we're enforcing style
checks or
>> >>>> > > we're not, and right now it seems we aren't.
>> >>>> > >
>> >>>> > > --
>> >>>> > > Marcelo
>> >>>> > >
>> >>>> > >
>> >>>> > >
>> ---------------------------------------------------------------------
>> >>>> > > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>> >>>> > >
>> >>>> > >
>> >>>> >
>> >>>>
>> >>>> ---------------------------------------------------------------------
>> >>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>> >>>>
>> >>>
>> >>
>> >
>> >
>> >
>> > --
>> > Marcelo
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>> >
>>
>>
>>
>> --
>> Marcelo
>>
>
>

Mime
View raw message