- Home
- Skills
- Testing & Debugging
- Regression test check
Regression test check
Verify that regression tests accurately distinguish between buggy and fixed code behavior.
Free
Works with the AI Agents you already use
Regression test check
Example session with this skill installed
Check if the new test in tests/auth_test.py correctly catches the reported session timeout bug where tokens were expiring 5 minutes too early.
- Read your context and instructions
- Compiled the regression test check
I have analyzed the candidate test tests/auth_test.py against both the original and fixed revisions. I traced the session initialization and verified that the test reaches the expiration logic without being intercepted by mocks.
Regression Test Verdict
| Metric | Observation | Result |
|---|---|---|
| Test Path | tests/auth_test.py::test_session_timeout_logic | Validated |
| Original Behavior | Fails with AssertionError: 2700 != 3000 | Verified Bug |
| Fixed Behavior | Passes (Session duration 3000s) | Verified Fix |
| Assertion Strength | Checks specific duration value, not just truthiness | Strong |
| Execution Evidence | Identical test run via pytest in both worktrees | Confirmed |
Analysis
The test correctly reproduces the regression. In the original revision, the session manager was incorrectly subtracting a 300-second buffer twice. The test uses a fixed clock mock to check the exact expiration timestamp, ensuring the behavior is not masked by system time jitter. The assertion assert session.expires_at == expected_time is sufficiently narrow to prevent false passes.
Next steps
- Merge the test case as part of the current PR.
- Monitor the CI pipeline to ensure the new test doesn't conflict with global fixture timeouts.
- Consider adding a boundary case for sessions starting exactly at midnight to check for edge-case offsets.
Connects securely to your tools. The creator never sees your data.
About this skill
Stop relying on suite-wide passes to confirm bug fixes. This skill validates whether a specific regression test actually distinguishes faulty behavior from the intended fix. It prevents the common pitfall of false positives caused by weak assertions, environmental failures, or bypassed code paths.
What it does
- Assertion validation checks if test assertions are specific enough to fail on the bug and pass on the fix.
- Trace analysis identifies if mocks or setups bypass the actual code path being tested.
- Side-by-side comparison runs the exact same test against both original and fixed revisions to ensure consistency.
- Failure diagnosis distinguishes between genuine bug reproduction and unrelated environment or fixture errors.
- Static review provides logic audits when execution environments are unavailable or blocked.
How it works
- Identify and trace the candidate test, mapping out setup, execution flow, and specific assertions.
- Isolate revisions using temporary worktrees or copies to preserve existing workspace state while testing different versions.
- Execute comparisons by running the identical test against the buggy and fixed code under matched configurations.
- Verify and report the verdict, citing specific locations and providing evidence of reproduction or failure causes.
Frameworks & tools
This skill is agnostic to testing frameworks. It works with standard runners like Jest, Pytest, JUnit, or Vitest, provided the agent has access to the repository and a command execution environment.
Why this beats prompting it yourself
Generic prompts often lead agents to assume a test works just because it fails. This skill enforces a strict verification protocol that distinguishes between "test failed because of the bug" and "test failed because the database didn't start." It eliminates false confidence by requiring evidence of both failure on the bug and success on the fix using identical test logic.
Use cases
- Reviewing a PR's new regression test to ensure it isn't a "shallow" pass.
- Diagnosing why a test remains green despite a known bug being present in the code.
- Verifying bug fix efficacy when multiple revisions are involved.
- Auditing mocks to ensure they aren't hiding regressions.
Known limitations
Requires access to the codebase and, for full verification, a working execution environment for the test suite. Static analysis cannot guarantee runtime behavior.
Requires access to the codebase and a functional test runner for dynamic verification. Static analysis is used as a fallback but cannot guarantee runtime results.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Free forever
No account required to browse
Trust & safety
Security scan
Security scan passed 16 days ago
- Free with an Agensi account