- Home
- Skills
- Testing & Debugging
- Flaky Test Triage Card
Works with the AI Agents you already use
Flaky Test Triage Card
Sort flaky failures into timing, state, order, network and nondeterminism, with the diagnostic command and the fix for each.
Free
Flaky Test Triage Card
Example session with this skill installed
Triage these failures: test_user_signup failed with 'timeout 5000ms' on worker 4 but passed on retry. test_delete_post passed alone but failed when run after test_create_post in the full suite.
- Read your context and instructions
- Compiled the flaky test triage
I have classified these failures based on the provided log patterns. The signup failure points to an async timing issue, while the post deletion failure indicates a clear order dependence.
Flaky triage — Backend Integration Suite — Last 24 Hours
Collected: 2 failures across 48 runs
| Test | Cause | Confidence | Evidence | Fix | Owner | Quarantine expiry |
|---|---|---|---|---|---|---|
test_user_signup | Timing / async | High | Timeout 5000ms; passed on retry | Replace sleep with wait_for_selector | @dev_alpha | 2023-11-20 |
test_delete_post | Test-order dependence | High | Passes in isolation; fails in suite | Ensure test_create_post cleans up DB | @dev_beta | 2023-11-18 |
Diagnostic commands
# Verify timing for signup
pytest tests/auth/test_signup.py --count=20
# Verify order dependence for post deletion
pytest tests/blog/test_posts.py -p no:randomly
Suite health
Flaky rate: 4.1% of runs · Tests quarantined: 2 · Oldest quarantine: 2023-11-18
Next steps
- Run the
pytest --count=20command to confirm the signup flake frequency. - Update the
test_create_postfixture to use a proper teardown or unique IDs. - Set a calendar reminder for the quarantine expiry dates.
Connects securely to your tools. The creator never sees your data.
About this skill
Retries hide flakiness and turn a signal into noise. This card tells you which kind of flaky you have, because each kind has a different one-line fix.
What you get
- Five causes with their log fingerprints: timing, shared state, test-order dependence, network, and nondeterminism in the code under test.
- The commands that split them apart in minutes - repeat a single test 50 times, then vary the worker count.
- Three rules that keep a suite honest: quarantine entries expire, retries are a measurement rather than a fix, fix the cheap ones first.
- A triage report with a confidence column, an owner, and a quarantine expiry date.
How it works
Collect failures over a window instead of from one red build. If a test fails 1-in-50 alone, it is timing or nondeterminism. If it only fails alongside others, it is state or order. That split removes most of the argument.
Who it is for
Anyone whose CI is intermittently red and whose team has started ignoring it.
Limits
It classifies what the logs show. If more than a third of failures are unknown, fix the logging before the tests.
Free starter. For automated clustering by cause, a cost model that ranks which flakes are worth fixing, and quarantine automation with expiry, see Flaky Test Triage: https://www.agensi.io/skills/flaky-test-triage
Requires failure logs and retry data. Cannot fix underlying product bugs or suites with zero isolation. Classification depends on log quality.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Free forever
No account required to browse
Trust & safety
Security scan
Security scan passed 2 days ago
- Free with an Agensi account
Requires