Crawler Summary

macos-computer-use answer-first brief

Control the macOS UI — take screenshots, click, type, drag, find UI elements, and run AppleScript. Use when asked to interact with the desktop, open apps, fill forms, click buttons, or automate any visual/UI task on this Mac. --- name: macos-computer-use description: "Control the macOS UI — take screenshots, click, type, drag, find UI elements, and run AppleScript. Use when asked to interact with the desktop, open apps, fill forms, click buttons, or automate any visual/UI task on this Mac." homepage: "https://github.com/siddkb/openclaw-skills" metadata: { "clawdbot": { "emoji": "🖥️", "requires": { "bins": ["python3", "osascript"], "pip": Published capability contract available. No trust telemetry is available yet. Last updated 3/1/2026.

Freshness

Last checked 3/1/2026

Best For

Contract is available with explicit auth and schema references.

Not Ideal For

macos-computer-use is not ideal for teams that need stronger public trust telemetry, lower setup complexity, or more explicit contract coverage before production rollout.

Evidence Sources Checked

editorial-content, capability-contract, runtime-metrics, public facts pack

Claim this agent
Agent DossierGitHubSafety: 89/100

macos-computer-use

Control the macOS UI — take screenshots, click, type, drag, find UI elements, and run AppleScript. Use when asked to interact with the desktop, open apps, fill forms, click buttons, or automate any visual/UI task on this Mac. --- name: macos-computer-use description: "Control the macOS UI — take screenshots, click, type, drag, find UI elements, and run AppleScript. Use when asked to interact with the desktop, open apps, fill forms, click buttons, or automate any visual/UI task on this Mac." homepage: "https://github.com/siddkb/openclaw-skills" metadata: { "clawdbot": { "emoji": "🖥️", "requires": { "bins": ["python3", "osascript"], "pip":

OpenClawself-declared

Public facts

6

Change events

1

Artifacts

0

Freshness

Mar 1, 2026

Verifiededitorial-contentNo verified compatibility signals

Published capability contract available. No trust telemetry is available yet. Last updated 3/1/2026.

Schema refs publishedTrust evidence available

Trust score

Unknown

Compatibility

OpenClaw

Freshness

Mar 1, 2026

Vendor

Siddkb

Artifacts

0

Benchmarks

0

Last release

Unpublished

Executive Summary

Key links, install path, and a quick operational read before the deeper crawl record.

Verifiededitorial-content

Summary

Published capability contract available. No trust telemetry is available yet. Last updated 3/1/2026.

Setup snapshot

git clone https://github.com/siddkb/macos-computer-use.git
  1. 1

    Setup complexity is LOW. This package is likely designed for quick installation with minimal external side-effects.

  2. 2

    Final validation: Expose the agent to a mock request payload inside a sandbox and trace the network egress before allowing access to real customer data.

Evidence Ledger

Everything public we have scraped or crawled about this agent, grouped by evidence type with provenance.

Verifiededitorial-content
Vendor (1)

Vendor

Siddkb

profilemedium
Observed Mar 1, 2026Source linkProvenance
Compatibility (2)

Protocol compatibility

OpenClaw

contractmedium
Observed Feb 24, 2026Source linkProvenance

Auth modes

api_key

contracthigh
Observed Feb 24, 2026Source linkProvenance
Artifact (1)

Machine-readable schemas

OpenAPI or schema references published

contracthigh
Observed Feb 24, 2026Source linkProvenance
Security (1)

Handshake status

UNKNOWN

trustmedium
Observed unknownSource linkProvenance
Integration (1)

Crawlable docs

6 indexed pages on the official domain

search_documentmedium
Observed Apr 15, 2026Source linkProvenance

Release & Crawl Timeline

Merged public release, docs, artifact, benchmark, pricing, and trust refresh events.

Self-declaredagent-index

Artifacts Archive

Extracted files, examples, snippets, parameters, dependencies, permissions, and artifact metadata.

Self-declaredGITHUB OPENCLEW

Extracted files

0

Examples

6

Snippets

0

Languages

typescript

Parameters

Executable Examples

bash

~/.openclaw/workspace/skills/macos-computer-use/scripts/setup.sh

text

python3 screenshot.py [--output PATH] [--window TITLE] [--region X Y W H]

text

python3 mouse.py <subcommand> [args]

  click X Y [--button left|right|middle] [--double]
  move  X Y [--duration SECS]
  scroll X Y [--dx CLICKS] [--dy CLICKS]
  drag  X1 Y1 X2 Y2 [--duration SECS]

text

python3 keyboard.py <subcommand> [args]

  type   TEXT [--interval SECS]    # Type a string character by character
  press  KEY [KEY ...]             # Press one or more keys in sequence
  hotkey KEY [KEY ...]             # Press keys simultaneously (e.g. cmd shift s)

text

python3 find_ui.py [--app APP_NAME] [--role ROLE] [--title TITLE] [--label LABEL] [--limit N]

text

python3 applescript.py -e 'AppleScript expression'
python3 applescript.py -f script.applescript

Docs & README

Full documentation captured from public sources, including the complete README when available.

Self-declaredGITHUB OPENCLEW

Docs source

GITHUB OPENCLEW

Editorial quality

ready

Control the macOS UI — take screenshots, click, type, drag, find UI elements, and run AppleScript. Use when asked to interact with the desktop, open apps, fill forms, click buttons, or automate any visual/UI task on this Mac. --- name: macos-computer-use description: "Control the macOS UI — take screenshots, click, type, drag, find UI elements, and run AppleScript. Use when asked to interact with the desktop, open apps, fill forms, click buttons, or automate any visual/UI task on this Mac." homepage: "https://github.com/siddkb/openclaw-skills" metadata: { "clawdbot": { "emoji": "🖥️", "requires": { "bins": ["python3", "osascript"], "pip":

Full README

name: macos-computer-use description: "Control the macOS UI — take screenshots, click, type, drag, find UI elements, and run AppleScript. Use when asked to interact with the desktop, open apps, fill forms, click buttons, or automate any visual/UI task on this Mac." homepage: "https://github.com/siddkb/openclaw-skills" metadata: { "clawdbot": { "emoji": "🖥️", "requires": { "bins": ["python3", "osascript"], "pip": ["pyautogui", "mss", "pillow", "atomacos"] } } }

macOS Computer Use

Gives OpenClaw full control of the macOS UI: screenshots, mouse, keyboard, UI element discovery, and AppleScript. Pairs with Claude's vision to create a complete computer-use loop.

Setup (run once)

~/.openclaw/workspace/skills/macos-computer-use/scripts/setup.sh

This installs Python dependencies and guides you through granting the two required macOS permissions:

  • Accessibility → System Settings > Privacy & Security > Accessibility → add your terminal / OpenClaw
  • Screen Recording → System Settings > Privacy & Security > Screen Recording → add your terminal / OpenClaw

Quick Reference

| Goal | Script | Example | |------|--------|---------| | See the screen | screenshot.py | python3 screenshot.py | | Capture a window | screenshot.py | python3 screenshot.py --window "Safari" | | Capture a region | screenshot.py | python3 screenshot.py --region 0 0 800 600 | | Left-click | mouse.py click | python3 mouse.py click 500 300 | | Right-click | mouse.py click | python3 mouse.py click 500 300 --button right | | Double-click | mouse.py click | python3 mouse.py click 500 300 --double | | Move mouse | mouse.py move | python3 mouse.py move 200 400 | | Scroll | mouse.py scroll | python3 mouse.py scroll 500 300 --dy -5 | | Drag | mouse.py drag | python3 mouse.py drag 100 200 400 200 | | Type text | keyboard.py type | python3 keyboard.py type "Hello world" | | Press a key | keyboard.py press | python3 keyboard.py press return | | Keyboard shortcut | keyboard.py hotkey | python3 keyboard.py hotkey cmd shift s | | Find a button | find_ui.py | python3 find_ui.py --app "Safari" --role AXButton --title "Back" | | List UI elements | find_ui.py | python3 find_ui.py --app "Finder" | | Run AppleScript | applescript.py | python3 applescript.py -e 'tell app "Safari" to activate' |

Typical Computer-Use Loop

  1. screenshot.py → examine the screen (Claude sees the image)
  2. Identify target coordinates or UI element
  3. mouse.py click X Y or keyboard.py type "..." to act
  4. screenshot.py again to verify result
  5. Repeat until task is complete

Script Reference

screenshot.py

python3 screenshot.py [--output PATH] [--window TITLE] [--region X Y W H]
  • Saves PNG and prints the file path to stdout
  • Default: full screen saved to /tmp/openclaw-screenshot-<timestamp>.png
  • --window matches by window title substring (case-insensitive)
  • --region crops to pixel rectangle: x y width height

mouse.py

python3 mouse.py <subcommand> [args]

  click X Y [--button left|right|middle] [--double]
  move  X Y [--duration SECS]
  scroll X Y [--dx CLICKS] [--dy CLICKS]
  drag  X1 Y1 X2 Y2 [--duration SECS]
  • Coordinates are screen pixels (0,0 = top-left)
  • Scroll: negative dy scrolls down, positive scrolls up

keyboard.py

python3 keyboard.py <subcommand> [args]

  type   TEXT [--interval SECS]    # Type a string character by character
  press  KEY [KEY ...]             # Press one or more keys in sequence
  hotkey KEY [KEY ...]             # Press keys simultaneously (e.g. cmd shift s)
  • Key names follow pyautogui conventions: return, escape, tab, space, cmd, ctrl, shift, alt, delete, up, down, left, right, f1..f12

find_ui.py

python3 find_ui.py [--app APP_NAME] [--role ROLE] [--title TITLE] [--label LABEL] [--limit N]
  • Returns JSON array of matching elements with: title, role, label, position {x,y}, size {w,h}
  • Use position center to compute click coordinates
  • Common roles: AXButton, AXTextField, AXStaticText, AXMenuItem, AXWindow, AXCheckBox, AXLink
  • Requires atomacos: pip install atomacos

applescript.py

python3 applescript.py -e 'AppleScript expression'
python3 applescript.py -f script.applescript
  • Runs via osascript, returns stdout
  • Useful for app-level control, menu commands, file operations

Safety Notes

  • Mouse/keyboard actions are immediate — there is no undo
  • Always take a screenshot before and after actions to verify
  • Prefer find_ui.py over hard-coded coordinates when possible — coordinates shift if windows move
  • For destructive actions (file deletion, form submission, purchases), confirm with the user first
  • pyautogui has a failsafe: moving the mouse to the top-left corner (0,0) raises an exception and halts execution

Permissions Troubleshooting

If screenshot returns a black image → Screen Recording not granted If mouse/keyboard does nothing → Accessibility not granted Run setup.sh to re-check permission status.

Contract & API

Machine endpoints, protocol fit, contract coverage, invocation examples, and guardrails for agent-to-agent use.

Verifiedcapability-contract

Contract coverage

Status

ready

Auth

api_key

Streaming

No

Data region

global

Protocol support

OpenClaw: self-declared

Requires: openclew, lang:typescript

Forbidden: none

Guardrails

Operational confidence: medium

Contract is available with explicit auth and schema references.
Trust confidence is not low and verification freshness is acceptable.
Invocation examples
curl -s "https://xpersona.co/api/v1/agents/siddkb-macos-computer-use/snapshot"
curl -s "https://xpersona.co/api/v1/agents/siddkb-macos-computer-use/contract"
curl -s "https://xpersona.co/api/v1/agents/siddkb-macos-computer-use/trust"

Reliability & Benchmarks

Trust and runtime signals, benchmark suites, failure patterns, and practical risk constraints.

Missingruntime-metrics

Trust signals

Handshake

UNKNOWN

Confidence

unknown

Attempts 30d

unknown

Fallback rate

unknown

Runtime metrics

Observed P50

unknown

Observed P95

unknown

Rate limit

unknown

Estimated cost

unknown

No benchmark suites or observed failure patterns are available.

Media & Demo

Every public screenshot, visual asset, demo link, and owner-provided destination tied to this agent.

Missingno-media
No screenshots, media assets, or demo links are available.

Related Agents

Neighboring agents from the same protocol and source ecosystem for comparison and shortlist building.

Self-declaredprotocol-neighbors
GITHUB_REPOSactivepieces

Rank

70

AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents

Traction

No public download signal

Freshness

Updated 2d ago

OPENCLAW
GITHUB_REPOScherry-studio

Rank

70

AI productivity studio with smart chat, autonomous agents, and 300+ assistants. Unified access to frontier LLMs

Traction

No public download signal

Freshness

Updated 5d ago

MCPOPENCLAW
GITHUB_REPOSAionUi

Rank

70

Free, local, open-source 24/7 Cowork app and OpenClaw for Gemini CLI, Claude Code, Codex, OpenCode, Qwen Code, Goose CLI, Auggie, and more | 🌟 Star if you like it!

Traction

No public download signal

Freshness

Updated 6d ago

MCPOPENCLAW
GITHUB_REPOSCopilotKit

Rank

70

The Frontend for Agents & Generative UI. React + Angular

Traction

No public download signal

Freshness

Updated 23d ago

OPENCLAW
Machine Appendix

Contract JSON

{
  "contractStatus": "ready",
  "authModes": [
    "api_key"
  ],
  "requires": [
    "openclew",
    "lang:typescript"
  ],
  "forbidden": [],
  "supportsMcp": false,
  "supportsA2a": false,
  "supportsStreaming": false,
  "inputSchemaRef": "https://github.com/siddkb/macos-computer-use#input",
  "outputSchemaRef": "https://github.com/siddkb/macos-computer-use#output",
  "dataRegion": "global",
  "contractUpdatedAt": "2026-02-24T19:41:39.009Z",
  "sourceUpdatedAt": "2026-02-24T19:41:39.009Z",
  "freshnessSeconds": 4420059
}

Invocation Guide

{
  "preferredApi": {
    "snapshotUrl": "https://xpersona.co/api/v1/agents/siddkb-macos-computer-use/snapshot",
    "contractUrl": "https://xpersona.co/api/v1/agents/siddkb-macos-computer-use/contract",
    "trustUrl": "https://xpersona.co/api/v1/agents/siddkb-macos-computer-use/trust"
  },
  "curlExamples": [
    "curl -s \"https://xpersona.co/api/v1/agents/siddkb-macos-computer-use/snapshot\"",
    "curl -s \"https://xpersona.co/api/v1/agents/siddkb-macos-computer-use/contract\"",
    "curl -s \"https://xpersona.co/api/v1/agents/siddkb-macos-computer-use/trust\""
  ],
  "jsonRequestTemplate": {
    "query": "summarize this repo",
    "constraints": {
      "maxLatencyMs": 2000,
      "protocolPreference": [
        "OPENCLEW"
      ]
    }
  },
  "jsonResponseTemplate": {
    "ok": true,
    "result": {
      "summary": "...",
      "confidence": 0.9
    },
    "meta": {
      "source": "GITHUB_OPENCLEW",
      "generatedAt": "2026-04-16T23:29:18.771Z"
    }
  },
  "retryPolicy": {
    "maxAttempts": 3,
    "backoffMs": [
      500,
      1500,
      3500
    ],
    "retryableConditions": [
      "HTTP_429",
      "HTTP_503",
      "NETWORK_TIMEOUT"
    ]
  }
}

Trust JSON

{
  "status": "unavailable",
  "handshakeStatus": "UNKNOWN",
  "verificationFreshnessHours": null,
  "reputationScore": null,
  "p95LatencyMs": null,
  "successRate30d": null,
  "fallbackRate": null,
  "attempts30d": null,
  "trustUpdatedAt": null,
  "trustConfidence": "unknown",
  "sourceUpdatedAt": null,
  "freshnessSeconds": null
}

Capability Matrix

{
  "rows": [
    {
      "key": "OPENCLEW",
      "type": "protocol",
      "support": "unknown",
      "confidenceSource": "profile",
      "notes": "Listed on profile"
    }
  ],
  "flattenedTokens": "protocol:OPENCLEW|unknown|profile"
}

Facts JSON

[
  {
    "factKey": "docs_crawl",
    "category": "integration",
    "label": "Crawlable docs",
    "value": "6 indexed pages on the official domain",
    "href": "https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fopenclaw%2Fskills%2Ftree%2Fmain%2Fskills%2Fasleep123%2Fcaldav-calendar",
    "sourceUrl": "https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fopenclaw%2Fskills%2Ftree%2Fmain%2Fskills%2Fasleep123%2Fcaldav-calendar",
    "sourceType": "search_document",
    "confidence": "medium",
    "observedAt": "2026-04-15T05:03:46.393Z",
    "isPublic": true
  },
  {
    "factKey": "vendor",
    "category": "vendor",
    "label": "Vendor",
    "value": "Siddkb",
    "href": "https://github.com/siddkb/openclaw-skills",
    "sourceUrl": "https://github.com/siddkb/openclaw-skills",
    "sourceType": "profile",
    "confidence": "medium",
    "observedAt": "2026-03-01T06:02:06.680Z",
    "isPublic": true
  },
  {
    "factKey": "protocols",
    "category": "compatibility",
    "label": "Protocol compatibility",
    "value": "OpenClaw",
    "href": "https://xpersona.co/api/v1/agents/siddkb-macos-computer-use/contract",
    "sourceUrl": "https://xpersona.co/api/v1/agents/siddkb-macos-computer-use/contract",
    "sourceType": "contract",
    "confidence": "medium",
    "observedAt": "2026-02-24T19:41:39.009Z",
    "isPublic": true
  },
  {
    "factKey": "auth_modes",
    "category": "compatibility",
    "label": "Auth modes",
    "value": "api_key",
    "href": "https://xpersona.co/api/v1/agents/siddkb-macos-computer-use/contract",
    "sourceUrl": "https://xpersona.co/api/v1/agents/siddkb-macos-computer-use/contract",
    "sourceType": "contract",
    "confidence": "high",
    "observedAt": "2026-02-24T19:41:39.009Z",
    "isPublic": true
  },
  {
    "factKey": "schema_refs",
    "category": "artifact",
    "label": "Machine-readable schemas",
    "value": "OpenAPI or schema references published",
    "href": "https://github.com/siddkb/macos-computer-use#input",
    "sourceUrl": "https://xpersona.co/api/v1/agents/siddkb-macos-computer-use/contract",
    "sourceType": "contract",
    "confidence": "high",
    "observedAt": "2026-02-24T19:41:39.009Z",
    "isPublic": true
  },
  {
    "factKey": "handshake_status",
    "category": "security",
    "label": "Handshake status",
    "value": "UNKNOWN",
    "href": "https://xpersona.co/api/v1/agents/siddkb-macos-computer-use/trust",
    "sourceUrl": "https://xpersona.co/api/v1/agents/siddkb-macos-computer-use/trust",
    "sourceType": "trust",
    "confidence": "medium",
    "observedAt": null,
    "isPublic": true
  }
]

Change Events JSON

[
  {
    "eventType": "docs_update",
    "title": "Docs refreshed: Sign in to GitHub · GitHub",
    "description": "Fresh crawlable documentation was indexed for the official domain.",
    "href": "https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fopenclaw%2Fskills%2Ftree%2Fmain%2Fskills%2Fasleep123%2Fcaldav-calendar",
    "sourceUrl": "https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fopenclaw%2Fskills%2Ftree%2Fmain%2Fskills%2Fasleep123%2Fcaldav-calendar",
    "sourceType": "search_document",
    "confidence": "medium",
    "observedAt": "2026-04-15T05:03:46.393Z",
    "isPublic": true
  }
]

Sponsored

Ads related to macos-computer-use and adjacent AI workflows.