- Home
- Skills
- Agents & Orchestration
- hands-off: drive Windows apps without your mouse
hands-off: drive Windows apps without your mouse
Let your AI agent click buttons, fill in fields, pick menus and read windows in Windows desktop programs while you keep working. No cursor moves and no focus is stolen, even on a covered window or another monitor.
Free
Works with the AI Agents you already use
hands-off: drive Windows apps without your mouse
Example session with this skill installed
Open Calculator on my second monitor and work out 12 + 30 without touching my mouse or taking my focus. I'm still typing in my editor.
- Read your context and instructions
- Compiled the hands-off
Summary
Calculator opened on monitor 2 without focus, 12 + 30 entered by AutomationId, result read back:
42. Your editor kept the focus the whole time.
What I did
| Step | Tool | Target | Result |
|---|---|---|---|
| 1 | launch | calc.exe, monitor=2 | Calculator opened on monitor 2, focus stayed on your editor |
| 2 | tree | Calculator, filter "Button" | buttons found by id: num1Button, num2Button, plusButton, num3Button, num0Button, equalButton |
| 3 | invoke | id=num1Button, id=num2Button | 12 entered |
| 4 | invoke | id=plusButton | + |
| 5 | invoke | id=num3Button, id=num0Button | 30 entered |
| 6 | invoke | id=equalButton | = |
| 7 | read | id=CalculatorResults | "Display is 42" |
Check
- The result was read from the Calculator display itself, not assumed from the clicks.
- No cursor movement and no foreground change during the actions.
Next steps
- Want this again later? I can save it as
recipes/calculator_add.jsonwith ${a} and ${b} as inputs, so it runs in one call (or from a scheduled task with no agent). - Say
closeand I'll close the Calculator I opened (only that one).
Connects securely to your tools. The creator never sees your data.
About this skill
Most desktop automation borrows your mouse and keyboard: the cursor jumps, the window you were typing in loses focus, and you have to sit still until the agent is done. hands-off does it the other way. It talks to programs through Windows UI Automation and window messages (never SendInput), so the agent can work on monitor 2 while you keep working on monitor 1.
This skill teaches the agent when and how to use the free, open-source hands-off MCP server (MIT).
What the agent learns
- The order of choice: the program's own API first (bpy, COM, CDP, a CLI), hands-off second, the real mouse only after asking you.
- The flow:
list_windows→tree(buttons, fields, ids) →invoke/type_text/menu/keys/click_xy→ always check withreadorcapture. - Recipes: explore a task once, freeze it into a JSON recipe, then run it in one call (or with no agent at all, from a scheduler).
- The measured traps: WinUI/UWP apps that grab focus, repeated ids in Open/Save dialogs, black captures, modal dialogs that block, Ctrl/Alt shortcuts that read the real keyboard.
What works (measured)
| Program | Result |
|---|---|
| Win32/MFC | full tree, menus by WM_COMMAND, fields, buttons |
| WinUI (new Notepad) | write, read, File > Save |
| UWP (Calculator) | buttons and display by AutomationId |
| Chrome/Edge/Electron | browser chrome by UIA; page clicks by click_xy without moving the cursor |
| Games, Photoshop canvas, Blender viewport | no (they read the real mouse); use the app's API |
Measured with a person using the PC: 0 of 290 actions took the focus away from them.
Requirements
- Windows 10/11, uv, and the hands-off server registered once:
claude mcp add -s user hands-off -- uvx --from git+https://github.com/MareAltaStudio/hands-off hands-off-mcp - Works with any agent that loads MCP servers (Claude Code, Cursor, Copilot...).
Demo video, docs and source: https://github.com/MareAltaStudio/hands-off
Windows 10/11 only. Programs that read the real mouse (games, the Photoshop canvas, the Blender viewport) can't be driven; use the app's own API. Web page content in Chrome/Edge is reached by click_xy, not by the UI tree. Minimized windows can't be captured (move them first). Ctrl/Alt shortcuts through keys fail in many programs. Requires the hands-off MCP server and uv.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Free forever
No account required to browse
Trust & safety
Security scan
Security scan passed 20 days ago
- Free with an Agensi account
Requires
Permissions
The skill drives the hands-off MCP server, which reads and acts on the Windows desktop windows the agent names (UI Automation and window messages). Terminal: `launch` starts programs the agent asks for, and recipes can run a python step. Files: recipes are read from JSON files, captures can be saved as images, and every acting call is logged locally to %LOCALAPPDATA%\hands-off\actions.log. Network: only when installing (server from GitHub, packages from PyPI); the server itself makes no network calls.