Regression test check

    1

    Verify that regression tests accurately distinguish between buggy and fixed code behavior.

    Free

    Security scanned

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    Regression test check

    Example session with this skill installed

    Check if the new test in tests/auth_test.py correctly catches the reported session timeout bug where tokens were expiring 5 minutes too early.

    • Read your context and instructions
    • Compiled the regression test check

    I have analyzed the candidate test tests/auth_test.py against both the original and fixed revisions. I traced the session initialization and verified that the test reaches the expiration logic without being intercepted by mocks.

    Regression Test Verdict

    MetricObservationResult
    Test Pathtests/auth_test.py::test_session_timeout_logicValidated
    Original BehaviorFails with AssertionError: 2700 != 3000Verified Bug
    Fixed BehaviorPasses (Session duration 3000s)Verified Fix
    Assertion StrengthChecks specific duration value, not just truthinessStrong
    Execution EvidenceIdentical test run via pytest in both worktreesConfirmed

    Analysis

    The test correctly reproduces the regression. In the original revision, the session manager was incorrectly subtracting a 300-second buffer twice. The test uses a fixed clock mock to check the exact expiration timestamp, ensuring the behavior is not masked by system time jitter. The assertion assert session.expires_at == expected_time is sufficiently narrow to prevent false passes.

    Next steps

    1. Merge the test case as part of the current PR.
    2. Monitor the CI pipeline to ensure the new test doesn't conflict with global fixture timeouts.
    3. Consider adding a boundary case for sessions starting exactly at midnight to check for edge-case offsets.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Stop relying on suite-wide passes to confirm bug fixes. This skill validates whether a specific regression test actually distinguishes faulty behavior from the intended fix. It prevents the common pitfall of false positives caused by weak assertions, environmental failures, or bypassed code paths.

    What it does

    • Assertion validation checks if test assertions are specific enough to fail on the bug and pass on the fix.
    • Trace analysis identifies if mocks or setups bypass the actual code path being tested.
    • Side-by-side comparison runs the exact same test against both original and fixed revisions to ensure consistency.
    • Failure diagnosis distinguishes between genuine bug reproduction and unrelated environment or fixture errors.
    • Static review provides logic audits when execution environments are unavailable or blocked.

    How it works

    1. Identify and trace the candidate test, mapping out setup, execution flow, and specific assertions.
    2. Isolate revisions using temporary worktrees or copies to preserve existing workspace state while testing different versions.
    3. Execute comparisons by running the identical test against the buggy and fixed code under matched configurations.
    4. Verify and report the verdict, citing specific locations and providing evidence of reproduction or failure causes.

    Frameworks & tools

    This skill is agnostic to testing frameworks. It works with standard runners like Jest, Pytest, JUnit, or Vitest, provided the agent has access to the repository and a command execution environment.

    Why this beats prompting it yourself

    Generic prompts often lead agents to assume a test works just because it fails. This skill enforces a strict verification protocol that distinguishes between "test failed because of the bug" and "test failed because the database didn't start." It eliminates false confidence by requiring evidence of both failure on the bug and success on the fix using identical test logic.

    Use cases

    • Reviewing a PR's new regression test to ensure it isn't a "shallow" pass.
    • Diagnosing why a test remains green despite a known bug being present in the code.
    • Verifying bug fix efficacy when multiple revisions are involved.
    • Auditing mocks to ensure they aren't hiding regressions.

    Known limitations

    Requires access to the codebase and, for full verification, a working execution environment for the test suite. Static analysis cannot guarantee runtime behavior.

    Requires access to the codebase and a functional test runner for dynamic verification. Static analysis is used as a fallback but cannot guarantee runtime results.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Free forever

    No account required to browse

    Trust & safety

    Security scan

    Security scan passed 16 days ago

    • Free with an Agensi account

    Permissions

    Read Files
    Terminal / Shell
    Listed16 days ago

    What's inside

    Frequently Asked Questions

    This skill performs a dual-revision check, running the same test against both the buggy and fixed code to ensure the test fails for the right reason and passes only when the logic is corrected.

    The skill is framework-agnostic and works with any standard runner like Pytest, Jest, or JUnit, as long as the agent has permissions to execute commands in your repository.

    While it performs best in a live execution environment to verify runtime behavior, the skill includes a static review mode to audit test logic and assertions even when environment access is restricted.

    You receive the complete skill configuration and documentation, allowing your agent to immediately begin auditing regression tests and identifying weak assertions or bypassed code paths.

    Yes, the skill specifically analyzes traces and setups to identify if mocks are masking bugs or preventing the test from reaching the actual code path intended for verification.

    New in Testing & Debugging