Why Did the Test Fail? A Triage Rulebook, Tested on Two LLMs

This story was originally published on HackerNoon at: https://hackernoon.com/why-did-the-test-fail-a-triage-rulebook-tested-on-two-llms.

Why did the test fail: the product, the test, the data or the environment? An evidence-based triage rulebook with rules and 21 labelled cases.

Check more stories related to programming at: https://hackernoon.com/c/programming.

You can also check exclusive content about #software-development, #software-testing, #qa, #test-automation, #llms, #flaky-tests, #ai-testing, #hackernoon-top-story, and more.

This story was written by: @yevhen-bozhenko. Learn more about this writer by checking @yevhen-bozhenko's about page,

and for more stories, please visit hackernoon.com.

A five-class taxonomy for deciding whether a failed test means the product, the test, the data or the environment is wrong, with the evidence rules that tell them apart — then checked by asking two LLMs to classify 21 labelled failures, with the rules and without.