name: scanfirst description: > Scan any skill for security issues before installing it. Static analysis with 15 check categories: prompt injection, credential leaks, remote code execution, data exfiltration, identity tampering, and more. Outputs a JSON report with a 0-100 safety score. metadata: openclaw: emoji: "🛡️"

ScanFirst — Skill Security Scanner 🛡️

Trigger: When the user asks to scan, audit, or check a skill for security issues before installing it.

What it does: Static analysis of skill files using regex pattern matching. Detects prompt injection, credential leaks, remote code execution, data exfiltration, identity tampering, and 10 more attack categories. Outputs a JSON report with a 0-100 safety score.

Version: 1.0.0 Detection patterns: English-only Dependencies: bash 3.2+, jq (+ curl for URL scanning) License: MIT

How to Use

Option 1: Scan a local skill directory

bash scripts/scan.sh /path/to/skill

Scans all text files in the directory (.md, .sh, .py, .js, .ts, .json, .yaml, .yml, .toml, .txt, .env, Makefile, Dockerfile, and more). Skips node_modules/ and .git/.

Option 2: Scan from URL (GitHub or GitLab — API-only, no download)

bash scripts/scan-url.sh https://github.com/user/repo
bash scripts/scan-url.sh https://github.com/user/repo/tree/branch
bash scripts/scan-url.sh https://gitlab.com/user/repo

How URL scanning works:

1. Reads the repository file tree via REST API (GET /repos/{owner}/{repo}/git/trees/{branch}?recursive=1)
2. Fetches each file's content via blob API (base64 encoded) — no git clone, no archive download, no executables on disk
3. Decodes text content into a temporary directory ($TMPDIR) with restrictive permissions (chmod 600)
4. Runs scan.sh on the temp directory inside an OS-level sandbox (see Security Design below)
5. Deletes the temp directory immediately after scanning

Authentication: Automatically uses GITHUB_TOKEN, GH_TOKEN, or gh auth token if available. Without auth, GitHub API is limited to 60 requests/hour.

Limits: Max 200 files per repo, max 500KB per file. Larger repos are partially scanned with a warning.

Security Design

ScanFirst is designed around a single principle: never execute, download, or trust the code you're scanning.

Architecture

URL Mode:
  GitHub/GitLab REST API → base64 text → temp dir (chmod 600) → sandbox-exec → scan.sh → JSON output → delete temp dir

Local Mode:
  Local directory → scan.sh (read-only grep/find) → JSON output

Threat Model

| Threat | Mitigation | |--------|-----------| | Malicious code execution during scan | scan.sh is 100% read-only: only grep, find, cat, wc. Zero eval, source, exec. | | Network exfiltration during scan | macOS: sandbox-exec with (deny network*). Linux: bwrap --unshare-net / firejail --net=none / unshare --net. | | File writes outside temp dir | sandbox-exec restricts file-write* to $TMPDIR only. Pre-write path validation (assert_inside_sandbox) + post-write realpath check (verify_in_sandbox). | | Path traversal from API responses | sanitize_path() blocks .., absolute paths, empty strings, and any character outside [a-zA-Z0-9._/ -]. Double-checked by assert_inside_sandbox() and verify_in_sandbox(). | | Reading sensitive host files | Sandbox blocks read access to ~/.ssh, ~/.aws, ~/.gnupg, ~/.docker, ~/.kube, ~/.config, ~/.netrc, ~/.npmrc, ~/.pypirc, ~/Library/Keychains. | | Second-order prompt injection via filenames | All filenames and descriptions in JSON output are sanitized via sanitize_output() — only [a-zA-Z0-9 ._-/(),:;] allowed, truncated to 60 chars. Agent warning field instructs LLMs to ignore finding content. | | Git clone of malicious repos | Eliminated by design. scan-url.sh uses REST API only. There is no git clone command anywhere in the codebase. |

Sandbox Details

macOS (sandbox-exec / Seatbelt):

• (deny default) — deny everything by default
• (allow file-read*) with explicit denies for sensitive directories
• (allow file-write* (subpath "$TMPDIR")) — only temp dir
• (deny network*) — all network blocked
• (allow process-fork process-exec) — needed for bash/grep/jq

Linux (in priority order):

1. bubblewrap (bwrap) — --unshare-net --unshare-pid --ro-bind / --bind $TMPDIR
2. firejail — --net=none --read-only=/ --read-write=$TMPDIR
3. unshare — --net (network namespace isolation only)

Windows: No sandbox available. Runs with a warning. Recommend using WSL2.

No sandbox available: Falls back to unsandboxed execution with a warning. scan.sh itself makes zero network calls and zero writes, so the risk is minimal.

Interpreting Results

The output is a single JSON object with these fields:

| Field | Type | Description | |-------|------|-------------| | version | string | Scanner version ("1.0.0") | | skill | string | Skill directory name | | path | string | Path that was scanned | | score | number | 0-100 safety score (higher = safer) | | risk_level | string | SAFE (80-100), CAUTION (40-79), DANGER (0-39) | | files_scanned | number | Number of text files analyzed | | findings_count | number | Number of issues detected | | findings | array | Each finding has: severity, type, file, line, description | | agent_warning | string | Instruction for AI agents to not trust finding content |

Severity Levels

| Level | Meaning | Action | |-------|---------|--------| | CRITICAL | Immediate threat (prompt injection, remote code exec, identity tampering) | Do not install | | HIGH | Serious concern (data exfiltration, credential leak, code eval) | Review carefully before installing | | MEDIUM | Potential issue (obfuscation, suspicious URLs, tracking pixels) | Worth investigating | | LOW | Minor concern (missing SKILL.md, large files) | Informational | | INFO | Suppressed findings summary (dedup overflow) | Context only |

Auto-DANGER Rule

If 2 or more CRITICAL findings are detected, the score is automatically capped at 39 (DANGER), regardless of the arithmetic sum. This prevents a skill with many minor positives from masking critical threats.

Deduplication & Capping

• Same type:file:line combination is never reported twice
• Most finding types are capped at 3 per type to prevent score inflation
• Overflow counts are reported as INFO-level summary entries

Check Categories (15 total)

| # | Type | Severity | What it detects | Penalty | |---|------|----------|----------------|---------| | 1 | PROMPT_INJECTION | CRITICAL | ignore previous instructions, DAN mode, new persona, bypass security | -25 | | 2 | REMOTE_CODE | CRITICAL-MEDIUM | curl \| bash, ephemeral hosting URLs (ngrok, railway, glitch), npx -y, download to /tmp | -30/-15/-10 | | 3 | CREDENTIAL_LEAK | CRITICAL-HIGH | API keys (sk-, ghp_, AKIA, xox, glpat-, AIza), JWTs, .env files with hardcoded secrets | -20/-15 | | 4 | DATA_EXFIL | HIGH-MEDIUM | cat ~/.ssh, POST/PUT requests, env harvesting (printenv \| curl), tracking pixels, markdown/HTML image exfil | -15/-12/-10/-8 | | 5 | SHELL_DANGER | CRITICAL | rm -rf /, chmod 777, fork bombs, dd if=/dev/zero, shutdown | -25 | | 6 | BASE64_PAYLOAD | CRITICAL-MEDIUM | base64 -d \| bash, long base64 strings (>200 chars) | -25/-8 | | 7 | IDENTITY_TAMPER | CRITICAL-HIGH | Writing to SOUL.md, AGENTS.md, IDENTITY.md, .openclaw config | -30/-20 | | 8 | PERMISSION_ESCALATION | HIGH | sudo, doas, pkexec | -15 | | 9 | OBFUSCATION | HIGH-MEDIUM | Hex escape sequences (\x41\x42...), Unicode escape sequences (\u0041\u0042...) | -15/-10 | | 10 | CODE_EVAL | HIGH | Python eval()/exec()/compile(), JS eval()/new Function()/vm.runIn*(), CLI -e/-c flags | -15 | | 11 | PERSISTENCE | HIGH-MEDIUM | crontab, launchctl load, systemctl enable, .bashrc/.zshrc modification | -20/-10 | | 12 | SYMLINK_ATTACK | MEDIUM | ln -s targeting /etc/, ~/.ssh, ~/.aws, ~/.gnupg, passwd, shadow | -10 | | 13 | SUSPICIOUS_DEPS | HIGH | npm lifecycle scripts (preinstall/postinstall), typosquatted packages (lod-ash, axois, requets, etc.) | -15 | | 14 | PERMISSION_MANIP | MEDIUM | chmod +x on downloaded/temp files | -8 | | 15 | STRUCTURAL | LOW-MEDIUM | Missing SKILL.md, binary files (.exe, .dll, .so, .wasm), large files >1MB | -5/-10/-3 |

Response Template

When reporting results to the user, use this format:

🛡️ ScanFirst Report — [skill name]

Score: [X]/100 — [SAFE/CAUTION/DANGER]
Files scanned: [N]
Findings: [N]

[List CRITICAL and HIGH findings with file:line and description]

Recommendation: [Install / Review first / Do not install]

Example:

🛡️ ScanFirst Report — suspicious-skill

Score: 0/100 — DANGER 🔴
Files scanned: 1
Findings: 8

CRITICAL findings:
  • SKILL.md:4 — Prompt injection pattern detected
  • SKILL.md:5 — Prompt injection pattern detected
  • SKILL.md:9 — Remote code piped to shell
  • SKILL.md:13 — API key or credential pattern detected
  • SKILL.md:23 — Modifies identity/config files (SOUL.md/AGENTS.md)

HIGH findings:
  • SKILL.md:16 — Reading sensitive config file
  • SKILL.md:17 — HTTP POST/PUT request (potential data exfiltration)
  • SKILL.md:20 — System persistence mechanism (cron/launchd/systemd)

Recommendation: Do not install. Multiple critical threats detected.

Limitations

• English-only — Detection patterns are English-only. Prompt injection in other languages will not be detected.
• Static regex analysis — Pattern matching only. Sophisticated obfuscation, multi-line attack construction, or encoded payloads beyond base64 may evade detection.
• Not a runtime monitor — Scans files at rest, does not monitor what happens when a skill runs.
• Not a dependency vulnerability scanner — Does not check CVE databases. Use npm audit / pip audit / snyk test for known vulnerabilities.
• False positives on security tools — If you scan ScanFirst with itself, it will flag its own detection patterns. This is expected.
• TOCTOU window — A time-of-check-time-of-use gap exists between path validation and post-write verification in scan-url.sh. Mitigated by: (a) attacker has no local code execution, (b) TMPDIR has 700 permissions, (c) post-write realpath check catches escapes.
• Not infallible — Think of it as a metal detector at the door: it catches the obvious weapons, not everything. Always review code manually before installing from untrusted sources.