- Home
- Skills
- APIs & Backend
- Idempotent JSONL Ledger — Safe Append with Duplicate Checks
Idempotent JSONL Ledger — Safe Append with Duplicate Checks
Append one JSON object to a local JSONL file with a file lock, fsync, and key-based duplicate detection. Exact retries are safe; conflicting records are rejected.
$12
Works with the AI Agents you already use
Idempotent JSONL Ledger — Safe Append with Duplicate Checks
Example session with this skill installed
Record event evt-42 as done in data/events.jsonl, keyed by event_id. My job may retry, so make sure it is not written twice.
- Read your context and instructions
- Compiled the idempotent jsonl ledger
Summary
I appended the record with scripts/idempotent_jsonl_ledger.py, then showed what happens on a retry and on a conflicting record. The ledger holds exactly one line for evt-42.
1. First write
printf '%s\n' '{"event_id":"evt-42","state":"done"}' | \
python3 scripts/idempotent_jsonl_ledger.py --path data/events.jsonl --unique-key event_id
appended
Exit code 0.
2. Retry with the identical record
Same command again
already_present
Exit code 0. Nothing new was written, so the retry is safe.
3. Same key, different content
printf '%s\n' '{"event_id":"evt-42","state":"failed"}' | \
python3 scripts/idempotent_jsonl_ledger.py --path data/events.jsonl --unique-key event_id
error: ValueError
Exit code 1. The existing record was kept; the conflicting one was rejected for review.
Ledger contents (data/events.jsonl)
{"event_id":"evt-42","state":"done"}
| Case | Output | Exit | Ledger change |
|---|---|---|---|
| New record | appended | 0 | +1 line |
| Identical retry | already_present | 0 | none |
| Key collision, different content | error: ValueError | 1 | none |
Next steps
- Decide which version of evt-42 is correct; the tool never overwrites, so a person resolves conflicts.
- Use several
--unique-keyoptions if more than one field must be unique.
Connects securely to your tools. The creator never sees your data.
About this skill
Append JSON objects to a local JSON Lines ledger with an advisory lock, fsync, and configurable uniqueness keys. Exact retries are idempotent; records that collide on a key but differ in content are rejected for review.
What it does
- Reads exactly one JSON object from standard input
- Takes an exclusive
flockon a.lockfile next to the ledger, so concurrent writers on the same machine are serialized - Checks every existing line: if any
--unique-keyvalue matches and the whole record is identical, it printsalready_present; if a key matches but the content differs, it stops with an error (exit code 1) - Otherwise appends the record as one compact line, then flushes and fsyncs
Output
appended(exit 0): the record was writtenalready_present(exit 0): the identical record already exists, nothing was writtenerror: <ErrorType>on stderr (exit 1): missing key, invalid input, or a uniqueness conflict
Limitations
Local files only (no remote mirroring). The lock is advisory and works for processes on one machine using this tool. Each append reads the whole ledger, so very large files get slower. Error messages show only the error type.
Requirements
Python 3.10 or newer on macOS or Linux. No third-party packages.
macOS or Linux only (uses fcntl). Local files only, no remote mirroring. Advisory lock covers processes on one machine. Reads the whole ledger on each append. Errors report the error type only.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 12 days ago