nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Witt <joe.w...@gmail.com>
Subject Re: [DISCUSS] CI / Travis / Jenkins
Date Tue, 05 Dec 2017 19:51:24 GMT
Andre

Thanks - read through https://issues.apache.org/jira/browse/NIFI-1657
where this was discussed and where the relevant multi-env commit came
in.

Seems like five environments may be too taxing based on the build
failures I'm observing.  I'll cut it down to three
FR
JP
US
For now.  We can evaluate if that helps at all and add more back if
things become stable.

Thanks
Joe

On Tue, Dec 5, 2017 at 12:20 AM, Andre <andre-lists@fucs.org> wrote:
> Joe,
>
> Glad to help! Few notes:
>
> If I recall correctly there was a reason we chose to add default and BR but
> to be honest I can't really remember what it was. I think it has to do with
> Time Zones + Locale issues and has helped detecting bizarre issues on time
> based junits (Matt B and Pierre may remember this).
>
> Regarding the rat check. The idea behind that was a fast failure in case of
> basic style violations, rather than wait until the end of the compilation.
> To be honest I don't know if this has worked as desired but should allow us
> to quickly identify validation errors which if I recall correctly were only
> detected at the end of contrib-check.
>
> And apologies for the anecdotal comments. I am away from my dev environment
> atm so I can't truly validate them.
>
>
> Kind regards
>
>
> On Tue, Dec 5, 2017 at 3:31 PM, Joe Witt <joe.witt@gmail.com> wrote:
>
>> Great news!  So for the first time in a long time we now have
>> travis-ci builds passing!
>>
>> I incorporated Dustin's PR which changed to the -Ddir-only instead of
>> -P, added Andre's idea of dropping the -quiet flag, and dropped the
>> number of builds in the config to a single parallel build with contrib
>> check now that we're seeing those pass with rat/checkstyle.
>>
>> https://travis-ci.org/apache/nifi/builds/311660398
>>
>> A couple failed due to test failures and I filed JIRAs to convert
>> these into integration tests or resolve.
>>  -https://issues.apache.org/jira/browse/NIFI-4660,
>> https://issues.apache.org/jira/browse/NIFI-4659
>>
>> One actually finished as you can see in its raw log but travis seems
>> to have gotten confused.
>>
>> Two passed completely.  I think to reduce strain on Travis-CI
>> infrastructure we should drop two of the environments.
>>
>> Current it is in .travis.yml
>>
>> env:
>>   - USER_LANGUAGE=en USER_REGION=US'
>>   - USER_LANGUAGE=fr USER_REGION=FR'
>>   - USER_LANGUAGE=ja USER_REGION=JP'
>>   - USER_LANGUAGE=pt USER_REGION=BR'
>>   - USER_LANGUAGE=default USER_REGION=default
>>
>> I think we should drop it to
>>
>> env:
>>   - USER_LANGUAGE=en USER_REGION=US'
>>   - USER_LANGUAGE=fr USER_REGION=FR'
>>   - USER_LANGUAGE=ja USER_REGION=JP'
>>
>> If no objections i'll do that soon.  But, good news is the builds are
>> coming back to life on Travis-CI and will help streamline review
>> cycles again!
>>
>> Thanks
>>
>> On Mon, Dec 4, 2017 at 8:29 PM, Joe Witt <joe.witt@gmail.com> wrote:
>> > nope. will take a look at this tonight though.
>> >
>> > On Dec 4, 2017 8:09 PM, "Andre" <andre-lists@fucs.org> wrote:
>> >>
>> >> Joe & Joey,
>> >>
>> >> I believe setting the maven compilation job to noisy - instead of the
>> >> current quiet setting - should help solving the issue.
>> >>
>> >> Have we tried that?
>> >>
>> >> Cheers
>> >>
>> >>
>> >> On 5 Dec 2017 6:26 AM, "Joe Witt" <joe.witt@gmail.com> wrote:
>> >>
>> >> I agree this would be extremely nice to get back on track.  The
>> >> changes made last night/today to the poms do appear to mean that
>> >> parallel builds with contrib-check are working.  Perhaps that helps us
>> >> a little with travis (or not).  I have reviewed a couple PRs though
>> >> recently that did not even compile much less have clean contrib-checks
>> >> so it is really nice to have Travis being more reliable.  Does anyone
>> >> have any sense of the current reasons for issues?  When I've looked
>> >> the errors made no sense at all.
>> >>
>> >> On Mon, Dec 4, 2017 at 2:21 PM, Joey Frazee <joey.frazee@icloud.com>
>> >> wrote:
>> >> > I’m sure everyone has noticed that Travis CI fails, incorrectly,
more
>> >> than it succeeds, often due to timeouts and not b/c of the incorrectness
>> >> of
>> >> a commit or PR.
>> >> >
>> >> > This has been discussed previously, but it’s carried on, and become
a
>> >> > low
>> >> information signal about the PRs, which has two big impacts: (1) it’s
>> >> ignored by experienced contributors and reviewers, and (2) it’s
>> confusing
>> >> or misleading to new contributors.
>> >> >
>> >> > So, we really need to find a solution. I can think of a few:
>> >> >
>> >> > 1. Continue to push on INFRA to setup Jenkins for NiFi and its
>> >> sub-projects.
>> >> >
>> >> > 2. Implement some kind of quick-test profile and shell script that
>> >> > checks
>> >> the most important things along with the subdirectories affected by the
>> >> PR,
>> >> and continue to use Travis CI.
>> >> >
>> >> > 3. Use some other service like Circle CI or Codeship, which probably
>> >> isn’t quite what ASF wants but it might make the CI more useful (it also
>> >> might not).
>> >> >
>> >> > 4. Find a sponsor to support a more premium tier of Travis CI (or
>> >> > equiv.)
>> >> so the build has enough resources to to succeed. This too probably isn’t
>> >> preferable but I’m sure we can find a precedent.
>> >> >
>> >> > I’m partial to pursuing (1) and (2) together because (1) would give
>> us a
>> >> long term solution and (2) would have some value for local builds (no
>> need
>> >> to run the full build) as well as making Travis CI tell us something.
>> The
>> >> first should be pretty low effort. The second will be labor intensive I
>> >> think — to identify what counts as quick and change the poms — so it
>> can’t
>> >> be the answer on its own unless we want to wait longer to see Travis CI
>> >> become informative.
>> >> >
>> >> > What do the rest of you think?
>> >> >
>> >> > -joey
>>

Mime
View raw message