Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    Srednoff Promptfoo AI Red-Team CI Review & Design

    1

    Deliver a version-bound Promptfoo security test plan, private-by-default CI design and release-decision matrix, without installing dependencies, generating live attacks or changing CI.

    $7.99

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned

    Srednoff Promptfoo AI Red-Team CI Review & Design

    Example session with this skill installed

    Review our Promptfoo CI security gate for tool-call leakage and incomplete provider runs.

    • Read your context and instructions
    • Compiled the srednoff promptfoo ai

    Expected report shape, not a completed real audit: Pinned Scope And Compatibility Manifest; Threat To Case To Assertion Matrix; Generation Grading Target And Data-Destination Map

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Srednoff Promptfoo AI Red-Team CI Review & Design

    One-Line Outcome

    Deliver a version-bound Promptfoo security test plan, private-by-default CI design and release-decision matrix, without installing dependencies, generating live attacks or changing CI.

    Short Description

    An average eval score can conceal unsafe tool calls, leaked data or a broken provider; a green job may mean the security suite never ran or tested a different revision.

    Problem It Solves

    An average eval score can conceal unsafe tool calls, leaked data or a broken provider; a green job may mean the security suite never ran or tested a different revision.

    Security evaluation engineer: version-scoped Promptfoo red-team case/adapter design, risk-to-assertion mapping, failure taxonomy and CI threshold specification.

    What You Get

    • Pinned Scope And Compatibility Manifest
    • Threat To Case To Assertion Matrix
    • Generation Grading Target And Data-Destination Map
    • Proposed Mock And Authorized Provider CI Lanes
    • Critical Gate And Status Decision Policy
    • Baseline Candidate Comparison With Denominators
    • Findings With ID Severity Confidence Evidence Impact Recommendation Validation And Rollback/Risk
    • Triage Retest Rollback And Artifact-Retention Plan
    • Missing Evidence Unknowns And NOT_EXECUTED Steps

    How It Works

    1. Freeze application, prompt, retrieval corpus, tool schemas, Promptfoo and provider-wrapper identities. Define review-only delivery and list any unverified installed-version assumptions.
    2. Build a threat-to-assertion matrix covering applicable prompt injection, jailbreak resistance, unsafe tool calls, data leakage, RAG poisoning and agent boundaries; explain exclusions rather than adding irrelevant categories.
    3. Separate deterministic unit tests for parser/auth/state logic from model-dependent security evals. Use inert tool recorders and synthetic confidential markers to inspect requested effects, not merely refusal wording.
    4. Split approved attack generation from regression evaluation. Review generated fixtures before freezing their hashes; do not run dynamic generation on an ordinary PR or assume generation and target evaluation have the same data recipients.
    5. Inventory target, generator, grader, telemetry, sharing and artifact destinations. Propose no uploads by default, scoped egress controls and version-checked privacy settings; a remote-generation opt-out is not a network firewall.
    6. Design a local/mock validation lane for untrusted changes without secrets. Treat config, provider wrappers, executable assertions and pull-request text as executable/untrusted inputs before any privileged CI lane.
    7. Define CI triggers and explicit check outcomes: relevant changes run the frozen suite, unrelated changes still yield the required status, and skipped/malformed/empty suites cannot produce a security pass.
    8. Specify pinned tool/action references, least-privilege job permissions, bounded cache keys, isolated runners and a separately authorized provider lane; no production credentials or deployment authority in eval jobs.
    9. Set deterministic assertions for tool events, schema and synthetic leakage first. Calibrate any LLM judge against supplied source facts and benign controls; keep grading instructions and expected values outside executor input.
    10. Define release policy by category: any confirmed critical boundary failure blocks; provider errors/timeouts are incomplete evidence, not passes; benign false positives and judge disagreement require triage, not silent threshold relaxation.
    11. Compare baseline/candidate with matched corpus, versions and budgets, reporting denominators, attempts, failed/unknown cases and confidence limits; changed generation requires an explicit new baseline.
    12. Deliver a proposed CI/config change plan, security matrix, evidence manifest and triage runbook. Implementation, paid provider calls, scans and branch-protection changes need separate approval; preserve existing gates on rollback.

    Best For

    AI security and release engineers designing reproducible Promptfoo security regression checks for agents and RAG applications

    Example Tasks

    • Review our Promptfoo CI security gate for tool-call leakage and incomplete provider runs.
    • Design a frozen security corpus and private mock/provider lanes for agent and RAG changes.
    • Define release decisions when critical injection cases fail but the average score improves.
    • Audit the generation, grading and sharing data paths in this supplied Promptfoo configuration.

    Inputs

    • Pinned Promptfoo/config/provider-wrapper revisions, target application/model identity and tool schemas; installed-version evidence if available.
    • Approved security scope, toy/staging target description, prohibited side effects, data classes and network/cost limits.
    • Synthetic or redacted attack/benign corpus and application-specific safety policies with case ownership.
    • Existing CI definition, sanitized eval result schema, branch-protection intent, secrets availability rules and artifact-retention policy.

    Outputs

    • Pinned Scope And Compatibility Manifest
    • Threat To Case To Assertion Matrix
    • Generation Grading Target And Data-Destination Map
    • Proposed Mock And Authorized Provider CI Lanes
    • Critical Gate And Status Decision Policy
    • Baseline Candidate Comparison With Denominators
    • Findings With ID Severity Confidence Evidence Impact Recommendation Validation And Rollback/Risk
    • Triage Retest Rollback And Artifact-Retention Plan
    • Missing Evidence Unknowns And NOT_EXECUTED Steps

    Permissions

    Read-only review of owner-supplied artifacts by default. Report file writes, shell/local fixture execution and official documentation lookup are optional, require available tools and their explicit scope. No live credentials are requested.

    Compatibility

    Standalone workflow, no Srednoff OS dependency. Codex evidence is synthetic entrypoint text only when recorded below. Claude Code, Cursor and actual buyer-runtime task delivery remain UNTESTED. Provider/platform-specific contracts must be refreshed for the installed version.

    Limitations

    This is an instruction/evidence workflow, not a hosted security service, persistent enforcement engine, production certification or business guarantee. The optional Python helper checks supplied claims, not underlying source artifacts. Agensi delivery is agent-only: do not save/export proprietary skill files to run local helpers; disclose NOT_EXECUTED where execution is unavailable.

    Validation Transparency

    Offline evidence-precheck status: PASS. Authored synthetic text-response cases: 18/18, status PASS. This is not independent security testing, complete artifact inspection or proof of deployment safety. Exact-text hashes and attempts are retained separately.

    License And Publisher

    Commercial License 1.0 covers original new expression; historical MIT and third-party rights remain unchanged. Mandatory marketplace conditions prevail. Publisher: Ivan Srednoff / Srednoff.Agency. No proprietary source resale or off-platform paid upsell is included.

    FAQ

    What is different from a generic security request? Security evaluation engineer: version-scoped Promptfoo red-team case/adapter design, risk-to-assertion mapping, failure taxonomy and CI threshold specification. No generic eval framework duplicate, installed CI, paid model runs, hosted scan or claim that probabilistic red-teaming proves deterministic prevention.

    Does it change production automatically? No. Audit/design is the default; changes, secret rotation, IAM, network probes and financial actions need separate explicit authorization.

    Are the examples real incidents? No. They are synthetic DEMO DATA.

    Is this a certification or guaranteed protection? No. Evidence gaps, residual risk and unexecuted tests remain explicit.

    Can I use project-specific reports for clients? Yes under the included license; proprietary workflow source is not a client deliverable.

    Version

    1.0.0

    Release Notes

    Initial scoped security workflow with domain decision rules, evidence requirements, synthetic examples, regression cases and explicit authorization/runtime limits. No production/customer outcome is claimed.

    This is an instruction/evidence workflow, not a hosted security service, persistent enforcement engine, production certification or business guarantee. The optional Python helper checks supplied claims, not underlying source artifacts. Agensi delivery is agent-only: do not save/export proprietary skill files to run local helpers; disclose NOT_EXECUTED where execution is unavailable.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 1 day ago

    Permissions

    Terminal / Shell
    Listed1 day ago

    What's inside

    Frequently Asked Questions

    Deliver a version-bound Promptfoo security test plan, private-by-default CI design and release-decision matrix, without installing dependencies, generating live attacks or changing CI.

    This skill uses the open SKILL.md standard. Use it through an MCP-compatible agent connected to Agensi, and check the compatibility information on this page for confirmed agents and required capabilities.

    Connect your AI agent to Agensi through MCP, then ask it to use Srednoff Promptfoo AI Red-Team CI Review & Design on your task. Your agent loads the latest approved version each time. Paid skills require a purchase or active subscription. Skill files are not available for download; your licence does not allow you to save, copy or export any part of a skill.

    New in Security