Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    Background Job Queue Reliability for BullMQ Celery Sidekiq

    1

    Stop background jobs being lost, run twice or stuck: idempotency, retries, dead letters, timeouts, graceful deploys, crash-tested. RELIABLE / AT RISK / LOSING JOBS.

    $5

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    Background Job Queue Reliability for BullMQ Celery Sidekiq

    Background Job Queue Reliability for BullMQ Celery Sidekiq

    Example session with this skill installed

    Audit our Celery setup. Django app, Redis broker, workers on Kubernetes. Tasks: send_invoice_email, charge_subscription, generate_report (up to 2 hours), sync_crm. Sometimes customers get two invoices and some reports never finish.

    • Read your context and instructions
    • Compiled the background job queue

    Job queue reliability: billing — VERDICT: LOSING JOBS

    Stack: Celery 5.5 on Redis, 4 pods × concurrency 8, terminationGracePeriodSeconds 30

    Job inventory

    JobSide effectMax runtimeRetriesTimeoutIdempotentAfter commit
    send_invoice_emailemail3 s3nonenono
    charge_subscriptionStripe charge4 s3nonenoyes
    generate_reportS3 file2 h0noneyesyes
    sync_crmHubSpot API20 s5noneyes (upsert)no

    Findings

    #SeverityJobRiskEvidenceFix
    1Criticalgenerate_reportRedis visibility timeout is 1 h (default); a 2 h task is redelivered and runs twice, and pods are killed after 30 s on deploy, so the report is lostsettings.py:212, no visibility_timeoutsplit into chunks with checkpoints; acks_late + task_reject_on_worker_lost; run on a dedicated queue
    2Highsend_invoice_emailenqueued inside atomic(); plus no idempotency, so a redelivery sends a second invoicebilling/services.py:88delay_on_commit() and an InvoiceEmailSent unique row per invoice
    3Highcharge_subscriptionretry can double-chargebilling/tasks.py:41pass Stripe idempotency_key=f"charge-{sub.id}-{period}"
    4Mediumallearly ack: pod OOM kill loses in-flight tasksdefault task_acks_late=Falseenable for idempotent tasks only
    5Mediumallno failed-task alertnoneexport task failures and queue length to your metrics; alert on failures > 0 in 10 min

    Tests run

    TestResult
    SIGKILL during send_invoice_email (after fix)redelivered, 1 email sent
    Rolled-back transactionno task ran
    charge retry with same idempotency key1 Stripe charge
    generate_report crash testnot run: needs staging data

    Verdict moves to AT RISK after fixes 1 to 3, and to RELIABLE once the report crash test passes in staging.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Background queues fail quietly. A Celery task is acknowledged before it runs and vanishes when the worker is killed. A BullMQ job runs longer than its lock and a second worker starts it again. A Sidekiq job is enqueued inside a transaction that rolls back. A deploy kills workers mid-job. Customers get two emails, two charges, or nothing at all.

    This skill gives your coding agent an audit procedure and a reference file of library defaults checked against the official docs. It:

    • Inventories every job: side effects, runtime, retries, timeouts, who enqueues it, and how workers stop on deploy
    • Checks 12 risks per job: idempotency, enqueue-after-commit, retries with backoff and a cap, dead-letter path, timeouts vs runtime, graceful shutdown, ack timing, payload size, duplicate schedulers, concurrency vs DB pool and API limits, retention, and monitoring
    • Knows the traps: Celery's early ack and the Redis visibility timeout with long ETAs, BullMQ's 30-second lock and close() with no timeout, Sidekiq's retry: false skipping the Dead set, Laravel's retry_after vs --timeout, SQS visibility and redrive
    • Fixes data-loss risks first, then duplicates, then operations, with small diffs
    • Proves the fixes with a crash test (SIGKILL mid-job), a graceful-stop test, retry and dead-letter tests, and a rolled-back transaction test
    • Writes a short runbook: replay dead jobs, pause a queue, what to alert on
    • Returns a verdict: RELIABLE, AT RISK or LOSING JOBS

    Works with: BullMQ, Celery, Sidekiq, RQ, Dramatiq, Oban, Laravel queues, Solid Queue, GoodJob, SQS and RabbitMQ.

    Safety: never purges, deletes or replays jobs in a real queue, and asks before changing production retry, timeout or concurrency settings.

    Honesty note: it quotes defaults from the reference file and asks you to confirm them against your installed version, and only reports tests it ran.

    Cannot replay or purge jobs in live queues; you run the provided commands. Library defaults must be confirmed against your installed version. Queues not listed get the generic 12-point checklist without library-specific defaults. Crash tests need a local or staging environment.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 29 days ago

    Requires

    Queue library
    Queue broker

    Permissions

    Network Access
    Listed29 days ago

    What's inside

    Frequently Asked Questions

    This skill specializes in preventing critical failures like double-charging or data loss by auditing your task lifecycle, ensuring idempotency, and fixing misconfigured grace periods that lead to silent job drops.

    It supports major frameworks across Node.js, Python, Ruby, PHP, and Elixir, and is compatible with brokers like Redis, RabbitMQ, SQS, and Postgres-backed queues.

    No complex setup is required; simply provide your job definitions, library versions, and broker configuration for the skill to begin its 12-point reliability assessment.

    A SKILL.md with the audit procedure, the 12 checks, the fix order and verdict rules, plus a reference file of library defaults and traps for BullMQ, Celery and Sidekiq, notes for RQ, Dramatiq, Oban, Laravel, Solid Queue, GoodJob, SQS and RabbitMQ, and patterns such as idempotency keys and the transactional outbox. Your agent produces the findings table, code diffs, test scripts and a short runbook.

    The reference file was checked against the official docs in October 2026 and notes where defaults differ between versions. Your agent identifies your installed version first and asks you to confirm any default it quotes against that version's docs, because defaults can change in new releases.

    For safety, this skill focuses on auditing, configuration, and remediation code; it does not perform live job purging or replaying in production environments.

    New in APIs & Backend