Idempotent JSONL Ledger — Safe Append with Duplicate Checks

    1

    Append one JSON object to a local JSONL file with a file lock, fsync, and key-based duplicate detection. Exact retries are safe; conflicting records are rejected.

    $12

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    Idempotent JSONL Ledger — Safe Append with Duplicate Checks

    Example session with this skill installed

    Record event evt-42 as done in data/events.jsonl, keyed by event_id. My job may retry, so make sure it is not written twice.

    • Read your context and instructions
    • Compiled the idempotent jsonl ledger

    Summary

    I appended the record with scripts/idempotent_jsonl_ledger.py, then showed what happens on a retry and on a conflicting record. The ledger holds exactly one line for evt-42.

    1. First write

    printf '%s\n' '{"event_id":"evt-42","state":"done"}' | \
      python3 scripts/idempotent_jsonl_ledger.py --path data/events.jsonl --unique-key event_id
    
    appended
    

    Exit code 0.

    2. Retry with the identical record

    Same command again

    already_present
    

    Exit code 0. Nothing new was written, so the retry is safe.

    3. Same key, different content

    printf '%s\n' '{"event_id":"evt-42","state":"failed"}' | \
      python3 scripts/idempotent_jsonl_ledger.py --path data/events.jsonl --unique-key event_id
    
    error: ValueError
    

    Exit code 1. The existing record was kept; the conflicting one was rejected for review.

    Ledger contents (data/events.jsonl)

    {"event_id":"evt-42","state":"done"}
    
    CaseOutputExitLedger change
    New recordappended0+1 line
    Identical retryalready_present0none
    Key collision, different contenterror: ValueError1none

    Next steps

    1. Decide which version of evt-42 is correct; the tool never overwrites, so a person resolves conflicts.
    2. Use several --unique-key options if more than one field must be unique.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Append JSON objects to a local JSON Lines ledger with an advisory lock, fsync, and configurable uniqueness keys. Exact retries are idempotent; records that collide on a key but differ in content are rejected for review.

    What it does

    • Reads exactly one JSON object from standard input
    • Takes an exclusive flock on a .lock file next to the ledger, so concurrent writers on the same machine are serialized
    • Checks every existing line: if any --unique-key value matches and the whole record is identical, it prints already_present; if a key matches but the content differs, it stops with an error (exit code 1)
    • Otherwise appends the record as one compact line, then flushes and fsyncs

    Output

    • appended (exit 0): the record was written
    • already_present (exit 0): the identical record already exists, nothing was written
    • error: <ErrorType> on stderr (exit 1): missing key, invalid input, or a uniqueness conflict

    Limitations

    Local files only (no remote mirroring). The lock is advisory and works for processes on one machine using this tool. Each append reads the whole ledger, so very large files get slower. Error messages show only the error type.

    Requirements

    Python 3.10 or newer on macOS or Linux. No third-party packages.

    macOS or Linux only (uses fcntl). Local files only, no remote mirroring. Advisory lock covers processes on one machine. Reads the whole ledger on each append. Errors report the error type only.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 12 days ago

    Permissions

    Terminal / Shell
    Read Files
    Listed12 days ago
    Updated5 days ago

    What's inside

    Frequently Asked Questions

    It appends one JSON object from standard input to a local JSONL file under a file lock, with fsync, and checks your chosen uniqueness keys so retries never create duplicates.

    If the whole record is identical, the tool prints `already_present`, writes nothing, and exits 0. If a key matches but other fields differ, it refuses with exit code 1 and leaves the existing line untouched.

    Yes for processes on the same machine that all use this tool: each one takes an exclusive flock on a .lock file before checking and appending. The lock is advisory, so other programs writing to the file directly are not blocked.

    Python 3.10 or newer on macOS or Linux. No third-party packages. It does not run on Windows because it uses fcntl file locking.

    Yes. Pass `--unique-key` several times; a match on any one of them counts as a collision. Every listed key must be present in the record, otherwise the append is refused.

    Three files: SKILL.md (instructions for the agent), README.md (usage and requirements), and scripts/idempotent_jsonl_ledger.py (the append tool). No third-party source code is bundled.

    New in APIs & Backend