ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Павлухин Иван <vololo...@gmail.com>
Subject Re: Brainstorm: Make TC Run All faster
Date Wed, 01 May 2019 05:39:45 GMT
Hi Dmitriy,

Thank you for adding staistics! With a good measure we can easily
compare different setups.

ср, 1 мая 2019 г. в 08:34, Павлухин Иван <vololo100@gmail.com>:
>
> Nikolay,
>
> > I thought by "faster" we all meant time between One push RunAll button and one get
results.
> > What else definition are possible here?
>
> But this "faster" depends on a current load. E.g. if we have a lot of
> idle slaves then incresead parallelism will allow a build to complete
> "faster". If we have many slaves busy than parallelism might "slow"
> down an execution. I predict that you are mostly interested in average
> metrics when TC has a build queue as it usually have on weekdays. If
> so then aggregatin other related jobs into "Build Apache Ignite" might
> speed execution of RunAll measured in wall clock time.
>
> And also I would like to return to a point I already mentioned earlier
> in other conversations. Our CI environment suites perfectly for an
> agile way of working. I suppose that we could try different setups
> almost freely and see if it gives a desired effect (and rollback if
> no). I am ok to go with Vyacheslav proposal. But I would like to
> observe some metrics showing that builds become running faster than
> before. Can we do this?
>
> вт, 30 апр. 2019 г. в 17:42, Dmitriy Pavlov <dpavlov@apache.org>:
> >
> > Hi, One more metric we can introduce is efficiency. Test run time/overall
> > time. The goal of launching RunAll is to obtain tests results.
> >
> > I've added more statistics to be displayed in the TC Bot, so now you may
> > click 'more' button near 'chain results' caption and you'll see sum of
> > statistics for all chain or individual suite. Latest RunAll gives the
> > following:
> > Overall time 57h 52m 41.188s (
> > - Build Net Time: 45h 10m 31.686s,
> > - Tests: 38h 58m 30.8s,
> > - Src. Update: 2h 29m 51.14s,
> > - Artifacts Publishing: 26m 38.295s,
> > - Dependencies Resolving: 9h 45m 1.809s,
> > - Timeouts: 3h 54m 16.071s)
> >
> > Currently, Efficiency is around 67-72%
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > пн, 29 апр. 2019 г. в 12:57, Nikolay Izhikov <nizhikov@apache.org>:
> >
> > > Ivan,
> > >
> > > I thought by "faster" we all meant time between One push RunAll button and
> > > one get results.
> > > What else definition are possible here?
> > >
> > > В Пн, 29/04/2019 в 12:04 +0300, Павлухин Иван пишет:
> > > > Nikolay,
> > > >
> > > > > Why we should imagine it?
> > > >
> > > > Only in order to understand what do we mean by "faster". My question was:
> > > >
> > > > 1st approach will take less agent time in sum. 2nd approach will
> > > > complete faster in wall clock time. And your main concern is "total
> > > > agent time". Am I right?
> > > >
> > > > пн, 29 апр. 2019 г. в 12:01, Nikolay Izhikov <nizhikov@apache.org>:
> > > > >
> > > > > > Let's imagine that we have an infinite number of agents
> > > > >
> > > > > Why we should imagine it?
> > > > > We don't have infinite number of agents.
> > > > >
> > > > > And we have several concurrent Run All.
> > > > >
> > > > >
> > > > > В Пн, 29/04/2019 в 11:50 +0300, Павлухин Иван пишет:
> > > > > > Vyacheslav,
> > > > > >
> > > > > > I finally figured out that "faster" means "total agent time".
> > > > > >
> > > > > > Let's imagine that we have an infinite number of agents. And
2
> > > approaches:
> > > > > > 1. Uber "Build Apache Ignite" containing all checks.
> > > > > > 2. Separate jobs for compilation, checkstyle and etc.
> > > > > >
> > > > > > 1st approach will take less agent time in sum. 2nd approach
will
> > > > > > complete faster in wall clock time. And your main concern is
"total
> > > > > > agent time". Am I right?
> > > > > >
> > > > > > пн, 29 апр. 2019 г. в 11:42, Vyacheslav Daradur <daradurvs@gmail.com
> > > >:
> > > > > > >
> > > > > > > Hi, Ivan,
> > > > > > >
> > > > > > > We are in the thread "Make TC Run All faster", so the main
benefit
> > > is
> > > > > > > to make TC faster :)
> > > > > > >
> > > > > > > Advantages:
> > > > > > > - 1 TC agent required instead of 4;
> > > > > > > - RunAll will be faster, in case of builds queue;
> > > > > > >
> > > > > > > Also, the "licenses" profile is included in the step of
a release
> > > > > > > build. I believe check-style also should be included.
> > > > > > >
> > > > > > > The generation of Javadocs is an optional step at preparing
the
> > > > > > > release, but its check on TC takes significant time in
case of the
> > > > > > > separate build.
> > > > > > >
> > > > > > > > > Returning to "Build Apache Ignite" it seems to
me that
> > > ideally, it can
> > > > > > >
> > > > > > > be hierarchical.
> > > > > > >
> > > > > > > I agree, all the checks may be set as a separate step in
the
> > > build's
> > > > > > > configuration. This helps with the main problem I'm trying
to
> > > solve -
> > > > > > > resolving of dependencies which takes the most time of
the builds.
> > > > > > >
> > > > > > > On Mon, Apr 29, 2019 at 11:24 AM Павлухин Иван
<
> > > vololo100@gmail.com> wrote:
> > > > > > > >
> > > > > > > > Vyacheslav, Maxim,
> > > > > > > >
> > > > > > > > Can we once again outline what benefits aggregated
"Build Apache
> > > > > > > > Ignite" performing various checks has comparing to
a modularized
> > > > > > > > approach in which separate builds perform separate
tasks?
> > > > > > > >
> > > > > > > > For example, modularized approach looks nice because
it is
> > > similar to
> > > > > > > > good practices in software development where we separate
> > > > > > > > responsibilities between different classes instead
of
> > > aggregating them
> > > > > > > > into a single class. And as usual multiple classes
works together
> > > > > > > > coordinating by a class from upper level. So, in fact
it is a
> > > > > > > > hierarchical structure.
> > > > > > > >
> > > > > > > > Returning to "Build Apache Ignite" it seems to me
that ideally
> > > it can
> > > > > > > > be hierarchical. There is a top level compilation
(assembly?)
> > > job but
> > > > > > > > it is always clear what tasks does it perform (check
style, check
> > > > > > > > license and other subjobs).
> > > > > > > >
> > > > > > > > пт, 26 апр. 2019 г. в 17:06, Maxim Muzafarov
<maxmuzaf@gmail.com
> > > >:
> > > > > > > > >
> > > > > > > > > Folks,
> > > > > > > > >
> > > > > > > > > +1 for merging all these suites into the single
one. All these
> > > suites
> > > > > > > > > (Build Apache Ignite, Javadoc, Licenses Header,
Checkstyle)
> > > required
> > > > > > > > > to be `green` all the time. So, we can consider
making them a
> > > part of
> > > > > > > > > build Apache Ignite procedure.
> > > > > > > > >
> > > > > > > > > Also, I'd suggest going deeper. We can try to
merge `Licenses
> > > Header`
> > > > > > > > > into the `Code style checker` [1]. This will
simplify the code
> > > > > > > > > checking process.
> > > > > > > > >
> > > > > > > > > [1] http://checkstyle.sourceforge.net/config_header.html
> > > > > > > > >
> > > > > > > > > On Fri, 26 Apr 2019 at 13:17, Vyacheslav Daradur
<
> > > daradurvs@gmail.com> wrote:
> > > > > > > > > >
> > > > > > > > > > Ivan, you are right, I meant to combine
them into one.
> > > > > > > > > >
> > > > > > > > > > Here is a build [1], with enabled profiles
(check-licenses,
> > > > > > > > > > checkstyle) and check of javadoc to show
the idea.
> > > > > > > > > >
> > > > > > > > > > Seems it takes ~15 minutes.
> > > > > > > > > >
> > > > > > > > > > [1]
> > > https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_ExperimentalBuildApacheIgniteJavadocLicensesHeaderCheckstyle&branch_IgniteTests24Java8=
> > > <default>;;
> > > > > > > > > >
> > > > > > > > > > On Fri, Apr 26, 2019 at 12:06 PM Павлухин
Иван <
> > > vololo100@gmail.com> wrote:
> > > > > > > > > > >
> > > > > > > > > > > Hi Vyacheslav,
> > > > > > > > > > >
> > > > > > > > > > > What do you mean by uniting?
> > > > > > > > > > >
> > > > > > > > > > > For me it looks like [Javadocs] and
[Check Code Style] are
> > > not so time
> > > > > > > > > > > consuming comparing to tests, are not
they? Do you suggest
> > > to combine
> > > > > > > > > > > mentioned 4 jobs into one? How long
will it run in a such
> > > case?
> > > > > > > > > > >
> > > > > > > > > > > чт, 25 апр. 2019 г. в 10:50,
Vyacheslav Daradur <
> > > daradurvs@gmail.com>:
> > > > > > > > > > > >
> > > > > > > > > > > > Hi Igniters,
> > > > > > > > > > > >
> > > > > > > > > > > > At the moment we have several
separated test suites:
> > > > > > > > > > > > * ~Build Apache Ignite~ _ ~10..20mins
> > > > > > > > > > > > * [Javadocs] _ ~10mins
> > > > > > > > > > > > * [Licenses Headers] _ ~1min
> > > > > > > > > > > > * [Check Code Style] _ ~7min
> > > > > > > > > > > > The most time of each build (except
Licenses Headers) is
> > > taken by
> > > > > > > > > > > > dependency resolving.
> > > > > > > > > > > >
> > > > > > > > > > > > Their main goal is a check that
the project is built
> > > properly.
> > > > > > > > > > > >
> > > > > > > > > > > > Also, profiles of [Javadocs],
[Licenses Headers] uses at
> > > the step of
> > > > > > > > > > > > preparing release (see DEVNOTES.txt)
that means they are
> > > important.
> > > > > > > > > > > >
> > > > > > > > > > > > I'd suggest uniting the builds,
this should reduce the
> > > time of tests
> > > > > > > > > > > > on ~15 minutes and releases agents.
> > > > > > > > > > > >
> > > > > > > > > > > > What do you think?
> > > > > > > > > > > >
> > > > > > > > > > > > On Tue, Nov 27, 2018 at 3:56 PM
Павлухин Иван <
> > > vololo100@gmail.com> wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > Roman,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Do you have some expectations
how faster "correlated"
> > > tests
> > > > > > > > > > > > > elimination will make Run
All? Also do you have a
> > > vision how can we
> > > > > > > > > > > > > determine such "correlated"
tests, can we do it
> > > relatively fast?
> > > > > > > > > > > > >
> > > > > > > > > > > > > But all in all, I am not
sure that reducing a group of
> > > correlated
> > > > > > > > > > > > > tests to only one test can
show good stability.
> > > > > > > > > > > > > пн, 26 нояб. 2018 г.
в 17:48, aplatonov <
> > > aplatonovv@gmail.com>:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > It should be noticed
that additional parameter
> > > TEST_SCALE_FACTOR was added.
> > > > > > > > > > > > > > This parameter with
ScaleFactorUtil methods can be
> > > used for test size
> > > > > > > > > > > > > > scaling for different
runs (like ordinary and
> > > nightly RunALLs). If someone
> > > > > > > > > > > > > > want to distinguish
these builds he/she can apply
> > > scaling methods from
> > > > > > > > > > > > > > ScaleFactorUtil in own
tests. For nightly test
> > > TEST_SCALE_FACTOR=1.0, for
> > > > > > > > > > > > > > non-nightly builds TEST_SCALE_FACTOR<1.0.
For
> > > example in
> > > > > > > > > > > > > > GridAbstractCacheInterceptorRebalanceTest
test
> > > ScaleFactorUtil was used for
> > > > > > > > > > > > > > scaling count of iterations.
I guess that
> > > TEST_SCALE_FACTOR support will be
> > > > > > > > > > > > > > added to runs at the
same time with RunALL (nightly)
> > > runs.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > --
> > > > > > > > > > > > > > Sent from:
> > > http://apache-ignite-developers.2346864.n4.nabble.com/
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > --
> > > > > > > > > > > > > Best regards,
> > > > > > > > > > > > > Ivan Pavlukhin
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > Best Regards, Vyacheslav D.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > Best regards,
> > > > > > > > > > > Ivan Pavlukhin
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Best Regards, Vyacheslav D.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Best regards,
> > > > > > > > Ivan Pavlukhin
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best Regards, Vyacheslav D.
> > > > > >
> > > > > >
> > > > > >
> > > >
> > > >
> > > >
> > >
>
>
>
> --
> Best regards,
> Ivan Pavlukhin



-- 
Best regards,
Ivan Pavlukhin

Mime
View raw message