ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Pavlov <dpav...@apache.org>
Subject [DISCUSSION] Fixed test order to reduce flakyness
Date Tue, 05 Feb 2019 14:40:17 GMT
Dear Ignite Developers,

The original idea came from our recent habr.ru post related to Apache
Ignite TeamCity Bot (for Russian native speakers, you can read an original
https://habr.com/ru/company/sberbank/blog/436070/#comment_19616976 )

It is a known phenomenon when tests have an influence on each other. The
simplest case when Ignite Native persistence is used, and not properly
cleared after a test run. This can make some test failed afterward.

So, what if we will set predictable, for example, alphabetical tests
execution order (maven-surefire-plugin/runOrder/alphabetical). This may
have the following effect: the set of tests failed because of being
affected by the previous run will be constant, will be exactly the same
each run.

At some point, when we stabilize flaky tests enough, we may select random
order, but for now, this solution seems valid to me.

What do you think?

Sincerely,
Dmitriy Pavlov

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message