- Home
- Skills
- Workflow Automation
- Bot Screen Control
Bot Screen Control
Drives an X11 desktop with real mouse and keyboard: screenshots, clicks, drags, typing, no OCR.
Free
Works with the AI Agents you already use
About this skill
Drive a headless X11 desktop (Xfce on Xvnc, or any X11/XWayland session) with real pointer and keyboard events from a single Python script. No AT-SPI, no accessibility tree, no OCR, no MCP server: look at a screenshot, click a coordinate, verify with another screenshot.
Built for the case every vision-based approach breaks: SPA login walls, canvas apps, native windows and a desktop whose accessibility stack is missing or lying.
Commands
shot— capture, with--grid Nfor a labelled coordinate grid and cursor crosshairclick/dclick/move— with eased pointer glides and--expect-change(reports whether the screen actually changed)drag,scroll,type(Unicode via clipboard),fill X Y "text"(click, select-all, clear, type)key,windows(list/raise/close/focus),wait-change,wait-stablerecord/record-review— one frame per tick, lists frames where the screen changedcheck— environment self-check before the first real action
Method it enforces: screenshot → locate → act → verify with a new screenshot. Every command prints one line of JSON (--human for text), so results stay machine-readable inside an agent loop.
Requires Linux with an X11 session (XWayland windows work, native Wayland windows do not), ffmpeg, xdotool, and Python 3.10+ with Pillow. xclip for clipboard-based Unicode typing, setxkbmap for layout switching.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Free forever
No account required to browse
Trust & safety
Security scan
Security scan passed today
- Free with an Agensi account
Requires