shadow agent inventory audit

Engineering teams are losing visibility into the AI agents, MCP servers, and LLM-framework scaffolds their developers are committing to source control. This skill provides a static, file-based inventory sweep to identify unsanctioned AI tooling across an entire GitHub or GitLab organization. It solves the "shadow AI" problem by scanning repositories for specific configurations like .mcp.json, LangGraph scaffolds, and agent-invoking CI jobs that bypass traditional governance.

What it does

• Multi-repo scanning identifies agent configurations and framework scaffolds across all accessible organization repositories.
• Signal verification reads file contents to filter out false positives from documentation examples, test fixtures, or vendored dependencies.
• Confidence tiering distinguishes between abandoned experimental configs and actively invoked agents corroborated by CI/cron signals.
• Coverage reporting generates a mandatory audit statement detailing scanned versus total repos and specific blind spots like gitignored local configs.
• Allowlist cross-referencing marks findings as either matching existing approvals or requiring immediate owner review.

How it works

1. Define the sweep scope by enumerating all repositories in the primary GitHub or GitLab namespace and identifying excluded categories like private or archived repos.
2. Execute pattern matching against the source control for MCP client configs, framework files like agents.yaml, and LLM-calling CI workflows.
3. Classify findings by opening each hit to verify it is a functional configuration rather than a code comment or tutorial file.
4. Generate the inventory as a consolidated report or automated script that dedupes findings and assigns confidence tiers.

Frameworks & tools

The skill targets MCP (Model Context Protocol), LangGraph, CrewAI, AutoGen, and OpenAI configuration standards. It integrates with GitHub and GitLab APIs for organizational-scale scanning.

Why this beats prompting it yourself

Generic prompts often fail to distinguish between a tutorial file and a production agent, leading to high false-positive rates. This skill enforces strict content-reading rules and provides a defensible audit methodology that accounts for repository-scale deduplication and CI-invocation signals.

Use cases

• Conduct a pre-audit discovery to see which teams are using unapproved MCP servers.
• Build a baseline inventory before migrating to an enterprise AI governance platform.
• Monitor for "config drift" where developers commit new agent scaffolds into existing repos.

Known limitations

Cannot detect agents wired in interactive sessions or configs stored in gitignored local directories. Static scanning does not monitor live network traffic or browser extensions.