hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yongjun Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HADOOP-11045) Introducing a tool to detect flaky tests of hadoop jenkins test job
Date Tue, 02 Sep 2014 02:21:21 GMT
Yongjun Zhang created HADOOP-11045:

             Summary: Introducing a tool to detect flaky tests of hadoop jenkins test job
                 Key: HADOOP-11045
                 URL: https://issues.apache.org/jira/browse/HADOOP-11045
             Project: Hadoop Common
          Issue Type: Improvement
          Components: build, tools
            Reporter: Yongjun Zhang
            Assignee: Yongjun Zhang

File this jira to introduce a tool to detect flaky tests of hadoop jenkins test jobs. 

I developed the tool on top of some initial work [~tlipcon] did. We find it quite useful.
With Todd's agreement, I'd like to push it to upstream so all of us can share (thanks Todd
for the initial work and support). I hope you find the tool useful.

This is a tool for hadoop contributors rather than hadoop users. Thanks [~tedyu] for the advice
to put to dev-support dir.

Description of the tool:

# Given a jenkins test job, this script examines all runs of the job done
# within specified period of time (number of days prior to the execution
# time of this script), and reports all failed tests.
# The output of this script includes a section for each run that has failed
# tests, with each failed test name listed.
# More importantly, at the end, it outputs a summary section to list all failed
# tests within all examined runs, and indicate how many runs a same test
# failed, and sorted all failed tests by how many runs each test failed in.
# This way, when we see failed tests in PreCommit build, we can quickly tell 
# whether a failed test is a new failure or it failed before, and it may just 
# be a flaky test.
# Of course, to be 100% sure about the reason of a failed test, closer look 
# at the failed test for the specific run is necessary.

This message was sent by Atlassian JIRA

View raw message