- Home
- Skills
- Agents & Orchestration
- AgentHand
AgentHand
Give your AI agent a hand on Windows: a resident local service that drives the browser via CDP and native apps via real mouse and clipboard — batched action scripts, scoped screenshots, observe-only by default.
$5
Works with the AI Agents you already use
AgentHand
Example session with this skill installed
Case 1 — Prove the hand works end to end: open Notepad, enter a test string, save it, and verify the result on disk — not just in the UI.
Case 2 — Send a message in the already-logged-in ChatGPT web session and capture the reply.
- Read your context and instructions
- Compiled the agenthand
Case 1 — Notepad end-to-end closure (verified on a real machine, 2026-11-11)
The agent sent one batched script: launch Notepad inside the interactive session → click the editor → paste_text "AstroHand closure test: HAND-OK" via the clipboard → open Save As → save as hand-test.txt.
Result: the agent read the file back from the filesystem: 31 bytes, content matching the requested string character for character. The core action segment (click → paste → save) took
3.35 seconds, and every step was written to the audit log with evidence-frame hashes.
Case 2 — Web send/reply round trip (same evening)
Actions: cdp_connect to the debug browser (separate profile) → find_click the composer → type_text via Input.insertText → key Enter.
Result: ChatGPT replied
"HAND-OK" verbatim, captured in one scoped evidence frame. One script out, structured results back — no screenshot-guessing loop.
Connects securely to your tools. The creator never sees your data.
About this skill
The last mile for AI agents: from saying to doing
A chat model can only talk — it has a brain, but no hands. AgentHand gives it one: a small resident service on your Windows PC that lets your agent see the screen, drive the mouse and keyboard, and reach inside the browser — then execute batched action scripts at native speed and report back structured results.
Instead of the agent guessing pixel-by-pixel over dozens of round trips, you send one script of actions; AgentHand executes the whole batch locally and returns structured results plus scoped, hash-stamped evidence frames. A task costs hundreds of tokens, not tens of thousands.
What it does
- Browser line (CDP): a pure-stdlib Chrome DevTools Protocol client (hand-written WebSocket, zero dependencies) that connects to a debug-enabled Edge/Chrome, navigates, finds and clicks elements across iframes, types via
Input.insertText, and attaches files to file inputs withDOM.setFileInputFiles— no system file dialog, ever. - Coordinate line (native apps): real mouse via Win32
SendInputand clipboard paste for native and self-drawn UIs that have no usable DOM or accessibility tree. - Eyes: heartbeat status (online / locked / foreground window / idle seconds), lock-screen detection, a live MJPEG screen stream for the owner, and scoped screenshots (
window/rect/selector) stamped with SHA-256 and a stale-frame flag. - Batch protocol: actions include
cdp_connect,navigate,find_click,type_text,set_files,key,click_at,paste_text,wait_for, andshot. Long scripts can run async with job polling; one call timeout triggers automatic CDP-bridge recovery.
How it works
agent ──HTTP (token-authenticated)──► agent_hand.py (resident service, interactive session)
client/agent_client.py ├─ state heartbeat / lock detection
├─ capture screen grab + MJPEG stream
├─ cdp stdlib WebSocket + minimal CDP (iframe-aware)
├─ actions batch executor (audit / observe-only / lock refusal)
└─ wininput clipboard paste + SendInput
owner's phone ──MJPEG──► /stream.mjpeg (live screen; never routed through the model)
The control plane is an HTTP API on port 18765 (configurable). The data plane (live screen) is separate and meant for the human owner — the agent only ever receives the scoped evidence frames it explicitly requested.
Security boundaries — stated plainly
This is a desktop-control tool. It can click, type, and paste on the machine it is installed on. That power is fenced in by design:
- Observe-only by default. With
observe_only=true(the default), every state-changing action is refused. You enable execution explicitly, after running the built-in acceptance checks — and you can switch it back at any time. - Token authentication on every endpoint. No token, no access (401), including the screen stream and status.
- Scoped screenshots only. A
shotmust name a window, rectangle, or element; unscoped full-screen grabs are rejected at validation. Windows whose titles suggest passwords/credentials are refused. - Honest lock state. When the PC is locked, input and screenshots return
LOCKED— no retries, no workarounds. - Full audit log. Every action lands in a JSONL audit trail with target, outcome, and evidence hashes.
- Kill switch. Desktop shortcuts (
Disable-AgentHand/Enable-AgentHand) stop and resume execution instantly; the service keeps reporting heartbeat while disabled. - Logins and CAPTCHAs are never handled by the agent. The owner signs in personally; the agent works inside the already-authenticated session.
Requirements & transparency
- Target machine: Windows, Python 3.11+ and Pillow — nothing else; the CDP/WebSocket client is standard-library only.
- The installer registers a logon-triggered scheduled task and generates a random access token, which the owner hands to their agent. No telemetry, no phone-home: the service only listens for the owner's agent.
- Windows only. No macOS or Linux support.
- Self-drawn UIs (e.g., WeChat, DingTalk) expose near-empty accessibility trees; AgentHand works on them through the coordinate line (screenshots + clicks + paste), which is reliable but coordinate-based — large UI changes may require re-aiming.
- Desktop/Electron apps generally do not accept a remote-debugging port, so the CDP line is unavailable for them; use the coordinate line or a UI-tree tool instead.
- Locked screen = no operation. The service reports
LOCKEDand refuses input until the owner unlocks. - Cross-origin iframe coordinate math is not composed yet; inside iframes the browser line falls back to a JS click (flagged in results).
- Screen stream is MJPEG (adaptive frame rate); H.264/WebRTC is on the roadmap.
- The owner must sign in to target sites personally — the agent never enters credentials or solves CAPTCHAs.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 5 days ago
Requires