Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+21 more

    Flash Attention Tuner

    1

    Swap transformer attention over to memory-efficient kernels: enable PyTorch's fused SDPA path, benchmark speed, verify numerical parity, and configure sequence-

    Free

    3 usesSecurity scanned
    Flash Attention Tuner

    Flash Attention Tuner

    Example session with this skill installed

    I'm hitting OOM on my 4096 seq length transformer. Convert my manual attention block to use fused Flash Attention kernels and benchmark the memory savings.

    • Read your context and instructions
    • Compiled the flash attention tuner

    Replaced manual QK^T implementation with F.scaled_dot_product_attention.
    Benchmark (4096 seq):

    • Manual: 45.2ms | 12.4GB VRAM
    • Fused: 12.8ms | 1.8GB VRAM
      Max Delta: 8.4e-3 (FP16).
      Verified backend: SDPBackend.FLASH_ATTENTION.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    The problem

    Training or inference on long sequences (512+ tokens) often triggers CUDA Out-of-Memory (OOM) errors. Manual attention implementation using explicit matrix multiplication and softmax operations is memory-intensive and creates significant bottlenecks in your GPU profile.

    What it does

    • Swaps manual attention blocks for PyTorch Scaled Dot Product Attention (SDPA) fused kernels.
    • Forces and verifies specific backends like Flash Attention or Memory Efficient kernels.
    • Benchmarks latency and peak memory usage before and after optimization.
    • Performs numerical parity checks to ensure bit-level agreement between implementations.
    • Configures sequence length, head dimensions, and causal masks for transformer architectures.

    Frameworks & tools

    PyTorch (2.2+), CUDA, NVIDIA Nsight Systems (nsys), and torch.nn.functional.

    Why this beats prompting it yourself

    Optimizing attention kernels requires specific knowledge of PyTorch backend constraints, such as head dimension limits and dtype requirements. This skill provides a structured verification workflow that ensures you are actually hitting the fused path rather than silently falling back to slow, memory-heavy implementations.

    Use cases

    • Fixing OOM errors in custom transformer architectures during training.
    • Optimizing inference speed for LLMs handling long context windows.
    • Extending the context length of existing models on limited VRAM.
    • Auditing attention performance via automated micro-benchmarking.

    Known limitations

    Requires CUDA-enabled GPUs; kernels are not available for CPU-only workloads. Optimal speedups are restricted to fp16 or bf16 dtypes and head dimensions under 256.

    Requires CUDA-capable hardware and PyTorch 2.2 or higher. Flash kernels are only available for fp16 and bf16 dtypes with head dimensions up to 256.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before any payment.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    3 uses

    Times used to date

    Free forever

    No account required to browse

    Trust & safety

    Security scan

    Security scan passed 3 months ago

    • Free with an Agensi account

    Permissions

    Terminal / Shell
    Listed3 months ago

    What's inside

    Frequently Asked Questions

    This skill automates the transition from standard matrix multiplication attention to PyTorch's Scaled Dot Product Attention (SDPA) kernels, drastically reducing VRAM usage and increasing throughput for long-sequence tasks.

    The skill is optimized for PyTorch 2.2+ environments on NVIDIA GPUs; it requires a CUDA-capable device and works best with Half Precision (fp16) or BFloat16 data types.

    You receive the integration logic to enable fused kernels, an automated benchmarking suite to measure speed gains, and a numerical parity script to ensure model accuracy remains identical after the swap.

    Unlike manual prompting, this skill includes a verification workflow that detects silent fallbacks, ensuring your model is actually using Flash Attention or Memory Efficient kernels rather than defaulting to slow, unoptimized paths.

    Yes, the skill is designed to be architecture-agnostic and can be applied to custom transformer blocks, provided they follow standard head dimension and causal masking conventions.

    Buyers receive lifetime access to the skill, including any updates made to support newer PyTorch versions or additional attention backends as they are released.