hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wei-Chiu Chuang <weic...@apache.org>
Subject Re: Fixing flaky tests in Apache Hadoop
Date Thu, 22 Oct 2020 21:03:05 GMT
Thanks for raising the issue, Akira and Ahmed,

Fixing flaky tests is a thankless job so I want to take this opportunity to
recognize the time and effort.

We will always have flaky tests due to bad tests or simply infra issues.
Fixing flaky tests will take time but if they are not addressed it wastes
everybody's time.

Recognizing this problem, I have two suggestions:

1. Other projects such as HBase have a tool to exclude flaky tests from
being executed. They track flaky tests and display them in a dashboard.
This will allow good tests to pass while leaving time for folks to fix
them. Or we could manually exclude tests (this is what we used to do at

2. Dedicate a community "Bug Bash Day" / "Fix it Day". We had a bug bash
day two years ago, and maybe it's time to repeat it again:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75965105 this
is going to be tricky as we are in a pandemic and most of the community are
working from home, unlike the last time when we can lock ourselves in a
conference room and force everybody to work :)


On Thu, Oct 22, 2020 at 12:14 PM Akira Ajisaka <aajisaka@apache.org> wrote:

> Hi Hadoop developers,
> Now there are a lot of failing unit tests and there is an issue to
> tackle this bad situation.
> https://issues.apache.org/jira/browse/HDFS-15646
> Although this issue is in HDFS project, this issue is related to all
> the Hadoop developers. Please check the above URL, read the
> description, and volunteer to dedicate more time to fix flaky tests.
> Your contribution to fixing the flaky tests will be really
> appreciated!
> Thank you Ahmed Hussein for your report.
> Regards,
> Akira
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message