- Home
- Skills
- Research & Analysis
- Chatologist: AI Chat History Audit
Works with the AI Agents you already use
Chatologist: AI Chat History Audit
Turn exported ChatGPT and Claude conversations into structured evidence about AI usage, value, automation and human judgment.
Free
Chatologist: AI Chat History Audit
Example session with this skill installed
Run the bundled synthetic OpenAI example with the audit question “Audit student learning and technical problem-solving with AI”, a 100% sample and baseline labeling. This is a pipeline smoke test, not an audit of real users.
- Read your context and instructions
- Compiled the chatologist
Synthetic smoke-test demonstration
This is the complete generated domain report from the bundled four-message example. Baseline labels are for testing only and do not support research or claims about real people.
Chatologist Report: student-learning-technical-problemsolving
Labeled messages: 4
In-scope or domain-adjacent messages: 3
Scope Distribution
| label | n | pct |
|---|---|---|
| IN_SCOPE | 3 | 75.0 |
| OUT_OF_SCOPE | 1 | 25.0 |
Task Distribution
| label | n | pct |
|---|---|---|
| DEBUG_OR_FIX | 1 | 33.3 |
| LEARN_OR_STUDY | 1 | 33.3 |
| ASK_EXPLAIN | 1 | 33.3 |
AI Role Distribution
| label | n | pct |
|---|---|---|
| TUTOR | 3 | 100.0 |
Labor Relation
| label | n | pct |
|---|---|---|
| LEARNING_AUGMENTATION | 3 | 100.0 |
Human Value Visible
| label | n | pct |
|---|---|---|
| PROBLEM_FRAMING | 3 | 100.0 |
Verification Burden
| label | n | pct |
|---|---|---|
| UNKNOWN | 3 | 100.0 |
Risk Domains
| label | n | pct |
|---|---|---|
| NONE | 3 | 100.0 |
Interpretation Limits
- This report is descriptive unless the user adds an explicit causal design.
- Sampling percentage and labeling quality determine how much confidence to place in the counts.
- Private raw message text should not be quoted externally without separate review.
Next steps
For a meaningful audit, review the generated codebook, label sampled batches with model or human reviewers, validate those labels, and regenerate the report. Review privacy findings before sharing.
Connects securely to your tools. The creator never sees your data.
About this skill
What do you actually use AI for, and where does your own judgment matter most? Chatologist helps you explore those questions using conversation exports you provide.
It normalizes OpenAI/ChatGPT, Anthropic/Claude and supported generic JSON, JSONL or ZIP exports into CSV, JSONL and SQLite. It then builds an editable codebook around your audit question, prepares sampled message batches for review, validates labels and generates reports and evidence tables.
Use it to investigate study habits, technical problem solving, workplace AI use, verification burden, or how AI supports and substitutes for parts of your work.
What you get
- A repeatable workflow from exported conversations to a structured audit.
- A custom domain codebook generated for your question and sampled data.
- Sampling controls to limit message volume and model-review cost.
- Label validation, summary statistics and Markdown reports.
- Evidence tables with paraphrased excerpts and a privacy-pattern scan.
- Synthetic examples, Python scripts, reference guides and tests.
Requirements and privacy
Python 3.10 or later and an agent with local file and command access, such as Codex or Claude Code. The Python scripts use the standard library and make no runtime HTTP calls. No backend, telemetry or account scraping is included.
Provide an export yourself. If you use a cloud model or agent to label batches, those samples may be processed by that provider under your settings. Review data before sharing. The privacy scan is a helper, not a guarantee of complete anonymization.
The optional baseline labeler is only a smoke-test helper. Meaningful audits require reviewing the generated codebook and using model or human reviewers to label and validate the sample. Findings are descriptive and depend on sampling and label quality.
Free and open source under the MIT license.
Requires a user-provided export and Python 3.10+. Baseline labels are smoke tests only. Review and refine the generated codebook, then use model or human reviewers for meaningful labels. Results depend on sampling and label quality; no time savings, causal impact or complete anonymization is established automatically. Cloud model labeling may send selected samples to your chosen provider.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Free forever
No account required to browse
Trust & safety
Security scan
Security scan passed 17 days ago
- Free with an Agensi account
Requires
Permissions
Reads only user-provided exports and bundled references. Writes normalized files, codebooks, review batches, validated labels, reports and privacy-scan results to the chosen local output folder. Python scripts make no HTTP calls; cloud model/agent review may process selected samples through the provider you choose. No account connection or credentials are required.