Works with the AI Agents you already use
analytics reproducibility and audit trail builder
This skill builds a run manifest and evidence bundle to turn raw metadata into a verifiable, deterministic record of an analytics or model run.
$2.99
analytics reproducibility and audit trail builder
Example session with this skill installed
Create a run manifest for run_id 'PROD-772' using code revision a1b2c3d. Dataset 'sales_v2' has checksum e3b0c442. Check if a re-run with tolerance 0.01 reproduced the accuracy metric of 0.945.
- Read your context and instructions
- Compiled the analytics reproducibility
The run manifest for PROD-772 has been generated. The script verified the integrity of 'sales_v2' and compared the re-run metrics against the original values.
Reproducibility and Audit Trail — PROD-772, 2023-10-27
1. Scope, sources and assumptions
- Sources: E1 (User prompt), E2 (Metadata provided for PROD-772).
- Permission Scope: Public/Internal access allowed.
- Assumptions: Code revision
a1b2c3dis the final version used in production (E2).
2. Run manifest
- Code Revision: a1b2c3d
- Seed: Not supplied (Information gap — organizational validation required)
- Parameters SHA-256: 8e323... (Computed)
| source_id | version | role | pin_status | sha256 |
|---|---|---|---|---|
| sales_v2 | 2.1.0 | input | pinned | e3b0c442... |
Manifest SHA-256: f721a...
3. Checks
| Check | Status | Detail |
|---|---|---|
| Dataset Approval | PASS | sales_v2 is approved by DataGov |
| Access Scope | PASS | Within Internal sensitivity limits |
| Leakage Check | PASS | No train/test overlap detected |
4. Reproducibility result
Verdict: reproduced_within_tolerance
| Output | Metric | Original | Re-run | Difference | Tolerance | Verdict |
|---|---|---|---|---|---|---|
| model_out | accuracy | 0.9450 | 0.9442 | 0.0008 | 0.0100 | PASS |
5. Leakage, claims and access findings
- Overlap: 0 records found between train/test splits.
- Claims: Accuracy claim of 0.945 is traceable to model_out (E2).
6. Evidence bundle
| Ref | Kind | Id | SHA-256 | Status |
|---|---|---|---|---|
| E2 | Dataset | sales_v2 | e3b0c442... | verified |
| E3 | Output | model_out | d6a12b8... | verified |
7. Limitations and items for human review
- Information gap: Random seed was not recorded.
- Management decision required: This release is a draft (requires_human_approval: true).
This record supports, and does not replace, review by accountable people.
Next steps
- Provide the random seed to complete the execution context.
- Obtain formal sign-off from the lead data scientist for the 'PROD-772' release.
- Archive the evidence bundle SHA-256 in the project repository.
Connects securely to your tools. The creator never sees your data.
About this skill
Complex analytical pipelines often lack the verifiable evidence required for rigorous audit or publication. This skill builds a run manifest and evidence bundle to turn raw metadata into a verifiable, deterministic record of an analytics or model run.
What it does
- Pins execution context by recording dataset versions, code revisions, random seeds, and environment parameters.
- Computes integrity signatures using SHA-256 checksums for both input datasets and output artifacts to form a verifiable evidence bundle.
- Validates reproducibility by comparing re-runs against originals within a stated tolerance, reporting exact differences in metrics.
- Detects data leakage by checking for train/test overlap and temporal leakage between dataset splits.
- Audits access and claims by verifying dataset approval status, access scope, and the traceability of specific metrics to outputs.
How it works
- Gather metadata from the user including run identifiers, code commits, dataset versions, and environment specs.
- Build request.json following a strict schema to capture all run parameters and declared checksums.
- Execute validation script using
scripts/run.pyto process the metadata, verify integrity, and check business rules. - Generate the manifest as a structured Markdown report containing the evidence bundle and reproducibility verdict.
Frameworks & tools
This skill relies on a self-contained Python script (scripts/run.py) and JSON schemas to handle data validation. It uses standard SHA-256 hashing for checksum verification and follows a deterministic methodology for audit trail generation.
Why this beats prompting it yourself
Manual prompts often fail to detect subtle data leakage or misinterpret checksum mismatches. This skill uses a hard-coded script to ensure every calculation is deterministic and verifiable, preventing the AI from "hallucinating" metric alignments or ignoring missing versions.
Use cases
- Generating an audit-ready manifest for a peer-reviewed research paper or regulatory submission.
- Verifying that a model re-trained in a new environment produces results within an acceptable tolerance.
- Creating an evidence bundle for an internal data science review to ensure no unauthorized datasets were used.
- Validating that specific claims made in a report are actually traceable to the underlying model outputs.
Known limitations
Does not perform data lineage mapping or privacy risk assessments. Does not execute actual pipelines or approve releases. Requires a specific JSON schema for inputs.
Does not map data lineage, assess privacy risks, or execute pipelines. Requires specific input schema. Release status remains a draft requiring human approval.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 3 days ago