As testers we know that having “more tests” isn’t automatically better. Tests have ongoing maintenance costs. Migrating a large test suite to a different technology or cleaning up legacy test suites is a time-consuming task. Plus: “more tests” also means slower feedback on new bugs, because the whole suite now takes longer to execute!
Fabian Streitel studied computer science at TU Munich and has been working for over 10 years to increase the impact of test automation for his customers. Over the years, Fabian has seen the testing processes of hundreds of companies and has helped improve them. He leads the Test Intelligence team and all research in that area at CQSE.
As testers we know that having “more tests” isn’t automatically better. Tests have ongoing maintenance costs. Migrating a large test suite to a different technology or cleaning up legacy test suites is a time-consuming task. Plus: “more tests” also means slower feedback on new bugs, because the whole suite now takes longer to execute!
At the same time, our gut feeling tells us that not all tests are equally valuable. So many of us have asked themselves: can’t we throw away the least valuable tests, to save maintenance costs without sacrificing product quality?
In computer science research, this problem is called “test suite minimization”: removing tests from a test suite but at the same time preserving its overall bug-finding power as best as possible. In this talk, I will show “AI Test Clustering”, a novel approach that can identify test cases that provide little to no value. I will cover the theoretical fundamentals of this technique and show how we applied it to industry test suites. I will cover how we evaluated its effectiveness and made sure we don’t accidentally throw away important test cases. Finally, I will show several strategies for quickly “spring-cleaning” your own project with little risk.
