stef-openclaw-skills/docs/web-automation.md

web-automation

Automated web browsing and scraping using Playwright-compatible CloakBrowser, with one-shot extraction and broader persistent automation under a single skill.

What this skill is for

• One-shot extraction from one URL with JSON output
• Automating web workflows
• Authenticated session flows (logins/cookies)
• Extracting page content to markdown
• Working with bot-protected or dynamic pages

Command selection

• Use node skills/web-automation/scripts/extract.js "<URL>" for one-shot extraction from a single URL
• Use npx tsx scrape.ts ... for markdown scraping modes
• Use npx tsx browse.ts ..., auth.ts, or flow.ts for interactive or authenticated flows
• Use node skills/web-automation/scripts/zillow-discover.js "<street-address>" or har-discover.js to resolve a real-estate listing URL from an address
• Use node skills/web-automation/scripts/zillow-photos.js "<listing-url>" or har-photos.js for real-estate photo extraction before attempting generic gallery automation

Messaging rule:

• For WhatsApp or similar chat-driven runs, prefer native web_search, web_fetch, and bounded browser actions over shelling out to helper scripts for every core step.
• Treat the dedicated Zillow/HAR scripts as local/manual helpers, regression checks, or non-chat fallbacks.
• If a messaging workflow needs a subprocess at all, reserve it for a single final delivery step rather than the whole assessment.

Requirements

• Node.js 20+
• pnpm
• Network access to download the CloakBrowser binary on first use or via preinstall

First-time setup

cd ~/.openclaw/workspace/skills/web-automation/scripts
pnpm install
npx cloakbrowser install
pnpm approve-builds
pnpm rebuild better-sqlite3 esbuild

Updating CloakBrowser

cd ~/.openclaw/workspace/skills/web-automation/scripts
pnpm up cloakbrowser playwright-core
npx cloakbrowser install
pnpm approve-builds
pnpm rebuild better-sqlite3 esbuild

System libraries (for OpenClaw Docker builds)

export OPENCLAW_DOCKER_APT_PACKAGES="ffmpeg jq curl libnss3 libatk1.0-0 libatk-bridge2.0-0 libcups2 libdrm2 libxkbcommon0 libxcomposite1 libxdamage1 libxfixes3 libxrandr2 libgbm1 libasound2"

Native module note

If pnpm install warns that build scripts were ignored for native modules such as better-sqlite3 or esbuild, run:

pnpm approve-builds
pnpm rebuild better-sqlite3 esbuild

Without this, helper scripts may fail before launch because the native bindings are missing.

Prerequisite check

Before running automation, verify the local install and CloakBrowser wiring:

cd ~/.openclaw/workspace/skills/web-automation/scripts
node check-install.js

If this fails, stop and fix setup before troubleshooting site automation.

Exec approvals allowlist

If OpenClaw keeps prompting for approval when running this skill, add a local allowlist for the main agent:

openclaw approvals allowlist add --agent main "/opt/homebrew/bin/node"
openclaw approvals allowlist add --agent main "/usr/bin/env"
openclaw approvals allowlist add --agent main "~/.openclaw/workspace/skills/web-automation/scripts/*.js"
openclaw approvals allowlist add --agent main "~/.openclaw/workspace/skills/web-automation/scripts/node_modules/.bin/*"

Verify with:

openclaw approvals get

Notes:

• If node lives somewhere else, replace /opt/homebrew/bin/node with the output of which node.
• If matching is inconsistent, replace ~/.openclaw/... with the full absolute path for the machine.
• Keep the allowlist scoped to the main agent unless there is a clear reason to widen it.
• Prefer file-based commands like node check-install.js, node zillow-photos.js ..., and node har-photos.js ... over inline node -e .... Inline interpreter eval is more likely to trigger approval friction.
• The same applies to zillow-discover.js and har-discover.js: keep discovery file-based, not inline.

Common commands

# Install / wiring check
cd ~/.openclaw/workspace/skills/web-automation/scripts
node check-install.js

# One-shot JSON extraction
node skills/web-automation/scripts/extract.js "https://example.com"

# Zillow listing discovery from address
node skills/web-automation/scripts/zillow-discover.js "4141 Whiteley Dr, Corpus Christi, TX 78418"

# HAR listing discovery from address
node skills/web-automation/scripts/har-discover.js "4141 Whiteley Dr, Corpus Christi, TX 78418"

# Zillow photo extraction
node skills/web-automation/scripts/zillow-photos.js "https://www.zillow.com/homedetails/..."

# HAR photo extraction
node skills/web-automation/scripts/har-photos.js "https://www.har.com/homedetail/..."

# Browse a page with persistent profile
npx tsx browse.ts --url "https://example.com"

# Scrape markdown
npx tsx scrape.ts --url "https://example.com" --mode main --output page.md

# Authenticate flow
npx tsx auth.ts --url "https://example.com/login"

# General natural-language browser flow
npx tsx flow.ts --instruction 'go to https://search.fiorinis.com then type "pippo" then press enter then wait 2s'

Real-estate listing discovery and photo extraction

Use the dedicated Zillow and HAR discovery/photo commands before trying a free-form gallery flow. Discovery is unit-aware when the address includes an apartment / unit / suite identifier, and still supports plain no-unit addresses for single-family homes.

Zillow discovery

cd ~/.openclaw/workspace/skills/web-automation/scripts
node zillow-discover.js "4141 Whiteley Dr, Corpus Christi, TX 78418"

What it does:

• opens the Zillow address URL with CloakBrowser
• resolves directly to a property page when Zillow supports the address slug
• otherwise looks for a homedetails listing link in the rendered page
• returns the discovered listing URL as JSON
• fails fast with a timeout if the browser-backed discovery stalls

Operational note:

• when imported by property-assessor, Zillow discovery is allowed a longer source-specific timeout than the generic helper default, because some exact-unit Zillow pages resolve more slowly than the basic search/listing flow

HAR discovery

cd ~/.openclaw/workspace/skills/web-automation/scripts
node har-discover.js "4141 Whiteley Dr, Corpus Christi, TX 78418"

What it does:

• opens the HAR address search page
• looks for a confident homedetail match in rendered results
• returns the discovered listing URL when HAR exposes a strong enough match
• returns listingUrl: null when HAR discovery is not confident enough
• fails fast with a timeout if the browser-backed discovery stalls

Zillow

cd ~/.openclaw/workspace/skills/web-automation/scripts
node zillow-photos.js "https://www.zillow.com/homedetails/4141-Whiteley-Dr-Corpus-Christi-TX-78418/2103723704_zpid/"

What it does:

• opens the listing page with CloakBrowser
• first checks whether the rendered listing shell already exposes a complete photo set in Zillow's embedded __NEXT_DATA__ payload
• if the visible See all XX photos count is missing, still trusts the embedded set when the page metadata confirms the count or when the embedded set is already clearly substantial
• only tries the See all photos / See all X photos entry point when the initial structured data is incomplete
• returns direct photos.zillowstatic.com image URLs as JSON
• fails fast with a timeout if the browser-backed extraction stalls

Operational note:

• when imported by property-assessor, Zillow photo extraction is allowed a longer source-specific timeout than the generic helper default, because some exact-unit Zillow listings expose the correct photo set only after a slower render path

Expected success shape:

• complete: true
• expectedPhotoCount matches photoCount
• imageUrls contains the listing photo set

Zillow identifiers

cd ~/.openclaw/workspace/skills/web-automation/scripts
node zillow-identifiers.js "https://www.zillow.com/homedetails/6702-Everhart-Rd-APT-T106-Corpus-Christi-TX-78413/2067445642_zpid/"

What it does:

• opens the Zillow listing shell without forcing the photo workflow
• inspects embedded __NEXT_DATA__ plus visible listing text
• extracts parcel/APN-style identifiers when Zillow exposes them
• returns those identifiers so property-assessor can use them as stronger CAD lookup keys than listing geo IDs

HAR

cd ~/.openclaw/workspace/skills/web-automation/scripts
node har-photos.js "https://www.har.com/homedetail/4141-whiteley-dr-corpus-christi-tx-78418/14069438"

What it does:

• opens the HAR listing page
• clicks Show all photos / View all photos
• extracts the direct pics.harstatic.com image URLs from the all-photos page
• fails fast with a timeout if the browser-backed extraction stalls

Expected success shape:

• complete: true
• expectedPhotoCount matches photoCount
• imageUrls contains the listing photo set

Test commands

From skills/web-automation/scripts:

node check-install.js
npm run test:photos
node zillow-discover.js "<street-address>"
node har-discover.js "<street-address>"
node zillow-photos.js "<zillow-listing-url>"
node har-photos.js "<har-listing-url>"

Use the live Zillow and HAR URLs above for a known-good regression check.

One-shot extraction (extract.js)

Use extract.js when the task is just: open one URL, render it, and return structured content.

Features

• JavaScript rendering
• lightweight stealth and bounded anti-bot shaping
• JSON-only output
• optional screenshot and saved HTML
• browser sandbox left enabled

Options

WAIT_TIME=5000 node skills/web-automation/scripts/extract.js "https://example.com"
SCREENSHOT_PATH=/tmp/page.png node skills/web-automation/scripts/extract.js "https://example.com"
SAVE_HTML=true node skills/web-automation/scripts/extract.js "https://example.com"
HEADLESS=false node skills/web-automation/scripts/extract.js "https://example.com"
USER_AGENT="Mozilla/5.0 ..." node skills/web-automation/scripts/extract.js "https://example.com"

Output fields

• requestedUrl
• finalUrl
• title
• content
• metaDescription
• status
• elapsedSeconds
• challengeDetected
• optional screenshot
• optional htmlFile

Persistent browsing profile

browse.ts, auth.ts, flow.ts, and scrape.ts use a persistent CloakBrowser profile so sessions survive across runs.

Canonical env vars:

• CLOAKBROWSER_PROFILE_PATH
• CLOAKBROWSER_HEADLESS
• CLOAKBROWSER_USERNAME
• CLOAKBROWSER_PASSWORD

Legacy aliases still supported for compatibility:

• CAMOUFOX_PROFILE_PATH
• CAMOUFOX_HEADLESS
• CAMOUFOX_USERNAME
• CAMOUFOX_PASSWORD

Natural-language flow runner (flow.ts)

Use flow.ts when you want a general command style like:

• "go to this site"
• "find this button and click it"
• "type this and press enter"

Example

npx tsx flow.ts --instruction 'go to https://example.com then click on "Sign in" then type "stef@example.com" in #email then press enter'

You can also use JSON steps for deterministic runs:

npx tsx flow.ts --steps '[{"action":"goto","url":"https://example.com"},{"action":"click","text":"Sign in"}]'