spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hyukjin Kwon <gurwls...@gmail.com>
Subject Re: Running lint-java during PR builds?
Date Tue, 22 May 2018 15:35:36 GMT
I opened a PR - https://github.com/apache/spark/pull/21399 to run it with
SBT.

2018-05-22 2:18 GMT+08:00 Reynold Xin <rxin@databricks.com>:

> Can we look into if there is a plugin for sbt that works and then we can
> put everything into one single builder?
>
> On Mon, May 21, 2018 at 11:17 AM Dongjoon Hyun <dongjoon.hyun@gmail.com>
> wrote:
>
>> Thank you for reconsidering this, Hyukjin. :)
>>
>> Bests,
>> Dongjoon.
>>
>>
>> On Mon, May 21, 2018 at 9:20 AM, Marcelo Vanzin <vanzin@cloudera.com>
>> wrote:
>>
>>> Is there a way to trigger it conditionally? e.g. only if the diff
>>> touches java files.
>>>
>>> On Mon, May 21, 2018 at 9:17 AM, Felix Cheung <felixcheung_m@hotmail.com>
>>> wrote:
>>> > One concern is with the volume of test runs on Travis.
>>> >
>>> > In ASF projects Travis could get significantly
>>> > backed up since - if I recall - all of ASF shares one queue.
>>> >
>>> > At the number of PRs Spark has this could be a big issue.
>>> >
>>> >
>>> > ________________________________
>>> > From: Marcelo Vanzin <vanzin@cloudera.com>
>>> > Sent: Monday, May 21, 2018 9:08:28 AM
>>> > To: Hyukjin Kwon
>>> > Cc: Dongjoon Hyun; dev
>>> > Subject: Re: Running lint-java during PR builds?
>>> >
>>> > I'm fine with it. I tried to use the existing checkstyle sbt plugin
>>> > (trying to fix SPARK-22269), but it depends on an ancient version of
>>> > checkstyle, and I don't know sbt enough to figure out how to hack
>>> > classpaths and class loaders when applying rules, so gave up.
>>> >
>>> > On Mon, May 21, 2018 at 1:47 AM, Hyukjin Kwon <gurwls223@gmail.com>
>>> wrote:
>>> >> I am going to open an INFRA JIRA if there's no explicit objection in
>>> few
>>> >> days.
>>> >>
>>> >> 2018-05-21 13:09 GMT+08:00 Hyukjin Kwon <gurwls223@gmail.com>:
>>> >>>
>>> >>> I would like to revive this proposal. Travis CI. Shall we give this
>>> try?
>>> >>> I
>>> >>> think it's worth trying it.
>>> >>>
>>> >>> 2016-11-17 3:50 GMT+08:00 Dongjoon Hyun <dongjoon@apache.org>:
>>> >>>>
>>> >>>> Hi, Marcelo and Ryan.
>>> >>>>
>>> >>>> That was the main purpose of my proposal about Travis.CI.
>>> >>>> IMO, that is the only way to achieve that without any harmful
>>> >>>> side-effect
>>> >>>> on Jenkins infra.
>>> >>>>
>>> >>>> Spark is already ready for that. Like AppVoyer, if one of you
files
>>> an
>>> >>>> INFRA jira issue to enable that, they will turn on that. Then,
we
>>> can
>>> >>>> try it
>>> >>>> and see the result. Also, you can turn off easily again if you
don't
>>> >>>> want.
>>> >>>>
>>> >>>> Without this, we will consume more community efforts. For example,
>>> we
>>> >>>> merged lint-java error fix PR seven hours ago, but the master
branch
>>> >>>> still
>>> >>>> has one lint-java error.
>>> >>>>
>>> >>>> https://travis-ci.org/dongjoon-hyun/spark/jobs/176351319
>>> >>>>
>>> >>>> Actually, I've been monitoring the history here. (It's synced
every
>>> 30
>>> >>>> minutes.)
>>> >>>>
>>> >>>> https://travis-ci.org/dongjoon-hyun/spark/builds
>>> >>>>
>>> >>>> Could we give a change to this?
>>> >>>>
>>> >>>> Bests,
>>> >>>> Dongjoon.
>>> >>>>
>>> >>>> On 2016-11-15 13:40 (-0800), "Shixiong(Ryan) Zhu"
>>> >>>> <shixiong@databricks.com> wrote:
>>> >>>> > I remember it's because you need to run `mvn install` before
>>> running
>>> >>>> > lint-java if the maven cache is empty, and `mvn install`
is pretty
>>> >>>> > heavy.
>>> >>>> >
>>> >>>> > On Tue, Nov 15, 2016 at 1:21 PM, Marcelo Vanzin <
>>> vanzin@cloudera.com>
>>> >>>> > wrote:
>>> >>>> >
>>> >>>> > > Hey all,
>>> >>>> > >
>>> >>>> > > Is there a reason why lint-java is not run during
PR builds? I
>>> see
>>> >>>> > > it
>>> >>>> > > seems to be maven-only, is it really expensive to
run after an
>>> sbt
>>> >>>> > > build?
>>> >>>> > >
>>> >>>> > > I see a lot of PRs coming in to fix Java style issues,
and
>>> those all
>>> >>>> > > seem a little unnecessary. Either we're enforcing
style checks
>>> or
>>> >>>> > > we're not, and right now it seems we aren't.
>>> >>>> > >
>>> >>>> > > --
>>> >>>> > > Marcelo
>>> >>>> > >
>>> >>>> > >
>>> >>>> > > ------------------------------------------------------------
>>> ---------
>>> >>>> > > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>> >>>> > >
>>> >>>> > >
>>> >>>> >
>>> >>>>
>>> >>>> ------------------------------------------------------------
>>> ---------
>>> >>>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>> >>>>
>>> >>>
>>> >>
>>> >
>>> >
>>> >
>>> > --
>>> > Marcelo
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>> >
>>>
>>>
>>>
>>> --
>>> Marcelo
>>>
>>
>>

Mime
View raw message