# web-automation

Automated web browsing and scraping using Playwright-compatible CloakBrowser, with one-shot extraction and broader persistent automation under a single skill.

## What this skill is for

- One-shot extraction from one URL with JSON output
- Automating web workflows
- Authenticated session flows (logins/cookies)
- Extracting page content to markdown
- Working with bot-protected or dynamic pages

## Command selection

- Use `node skills/web-automation/scripts/extract.js "<URL>"` for one-shot extraction from a single URL
- Use `npx tsx scrape.ts ...` for markdown scraping modes
- Use `npx tsx browse.ts ...`, `auth.ts`, or `flow.ts` for interactive or authenticated flows
- Use `node skills/web-automation/scripts/zillow-discover.js "<street-address>"` or `har-discover.js` to resolve a real-estate listing URL from an address
- Use `node skills/web-automation/scripts/zillow-photos.js "<listing-url>"` or `har-photos.js` for real-estate photo extraction before attempting generic gallery automation

Messaging rule:
- For WhatsApp or similar chat-driven runs, prefer native `web_search`, `web_fetch`, and bounded browser actions over shelling out to helper scripts for every core step.
- Treat the dedicated Zillow/HAR scripts as local/manual helpers, regression checks, or non-chat fallbacks.
- If a messaging workflow needs a subprocess at all, reserve it for a single final delivery step rather than the whole assessment.

## Requirements

- Node.js 20+
- `pnpm`
- Network access to download the CloakBrowser binary on first use or via preinstall

## First-time setup

```bash
cd ~/.openclaw/workspace/skills/web-automation/scripts
pnpm install
npx cloakbrowser install
pnpm approve-builds
pnpm rebuild better-sqlite3 esbuild
```

## Updating CloakBrowser

```bash
cd ~/.openclaw/workspace/skills/web-automation/scripts
pnpm up cloakbrowser playwright-core
npx cloakbrowser install
pnpm approve-builds
pnpm rebuild better-sqlite3 esbuild
```

## System libraries (for OpenClaw Docker builds)

```bash
export OPENCLAW_DOCKER_APT_PACKAGES="ffmpeg jq curl libnss3 libatk1.0-0 libatk-bridge2.0-0 libcups2 libdrm2 libxkbcommon0 libxcomposite1 libxdamage1 libxfixes3 libxrandr2 libgbm1 libasound2"
```

## Native module note

If `pnpm install` warns that build scripts were ignored for native modules such as `better-sqlite3` or `esbuild`, run:

```bash
pnpm approve-builds
pnpm rebuild better-sqlite3 esbuild
```

Without this, helper scripts may fail before launch because the native bindings are missing.

## Prerequisite check

Before running automation, verify the local install and CloakBrowser wiring:

```bash
cd ~/.openclaw/workspace/skills/web-automation/scripts
node check-install.js
```

If this fails, stop and fix setup before troubleshooting site automation.

## Exec approvals allowlist

If OpenClaw keeps prompting for approval when running this skill, add a local allowlist for the main agent:

```bash
openclaw approvals allowlist add --agent main "/opt/homebrew/bin/node"
openclaw approvals allowlist add --agent main "/usr/bin/env"
openclaw approvals allowlist add --agent main "~/.openclaw/workspace/skills/web-automation/scripts/*.js"
openclaw approvals allowlist add --agent main "~/.openclaw/workspace/skills/web-automation/scripts/node_modules/.bin/*"
```

Verify with:

```bash
openclaw approvals get
```

Notes:
- If `node` lives somewhere else, replace `/opt/homebrew/bin/node` with the output of `which node`.
- If matching is inconsistent, replace `~/.openclaw/...` with the full absolute path for the machine.
- Keep the allowlist scoped to the main agent unless there is a clear reason to widen it.
- Prefer file-based commands like `node check-install.js`, `node zillow-photos.js ...`, and `node har-photos.js ...` over inline `node -e ...`. Inline interpreter eval is more likely to trigger approval friction.
- The same applies to `zillow-discover.js` and `har-discover.js`: keep discovery file-based, not inline.

## Common commands

```bash
# Install / wiring check
cd ~/.openclaw/workspace/skills/web-automation/scripts
node check-install.js

# One-shot JSON extraction
node skills/web-automation/scripts/extract.js "https://example.com"

# Zillow listing discovery from address
node skills/web-automation/scripts/zillow-discover.js "4141 Whiteley Dr, Corpus Christi, TX 78418"

# HAR listing discovery from address
node skills/web-automation/scripts/har-discover.js "4141 Whiteley Dr, Corpus Christi, TX 78418"

# Zillow photo extraction
node skills/web-automation/scripts/zillow-photos.js "https://www.zillow.com/homedetails/..."

# HAR photo extraction
node skills/web-automation/scripts/har-photos.js "https://www.har.com/homedetail/..."

# Browse a page with persistent profile
npx tsx browse.ts --url "https://example.com"

# Scrape markdown
npx tsx scrape.ts --url "https://example.com" --mode main --output page.md

# Authenticate flow
npx tsx auth.ts --url "https://example.com/login"

# General natural-language browser flow
npx tsx flow.ts --instruction 'go to https://search.fiorinis.com then type "pippo" then press enter then wait 2s'
```

## Real-estate listing discovery and photo extraction

Use the dedicated Zillow and HAR discovery/photo commands before trying a free-form gallery flow.
Discovery is unit-aware when the address includes an apartment / unit / suite identifier, and still supports plain no-unit addresses for single-family homes.

### Zillow discovery

```bash
cd ~/.openclaw/workspace/skills/web-automation/scripts
node zillow-discover.js "4141 Whiteley Dr, Corpus Christi, TX 78418"
```

What it does:
- opens the Zillow address URL with CloakBrowser
- resolves directly to a property page when Zillow supports the address slug
- otherwise looks for a `homedetails` listing link in the rendered page
- returns the discovered listing URL as JSON
- fails fast with a timeout if the browser-backed discovery stalls

Operational note:
- when imported by `property-assessor`, Zillow discovery is allowed a longer source-specific timeout than the generic helper default, because some exact-unit Zillow pages resolve more slowly than the basic search/listing flow

### HAR discovery

```bash
cd ~/.openclaw/workspace/skills/web-automation/scripts
node har-discover.js "4141 Whiteley Dr, Corpus Christi, TX 78418"
```

What it does:
- opens the HAR address search page
- looks for a confident `homedetail` match in rendered results
- returns the discovered listing URL when HAR exposes a strong enough match
- returns `listingUrl: null` when HAR discovery is not confident enough
- fails fast with a timeout if the browser-backed discovery stalls

### Zillow

```bash
cd ~/.openclaw/workspace/skills/web-automation/scripts
node zillow-photos.js "https://www.zillow.com/homedetails/4141-Whiteley-Dr-Corpus-Christi-TX-78418/2103723704_zpid/"
```

What it does:
- opens the listing page with CloakBrowser
- first checks whether the rendered listing shell already exposes a complete photo set in Zillow's embedded `__NEXT_DATA__` payload
- if the visible `See all XX photos` count is missing, still trusts the embedded set when the page metadata confirms the count or when the embedded set is already clearly substantial
- only tries the `See all photos` / `See all X photos` entry point when the initial structured data is incomplete
- returns direct `photos.zillowstatic.com` image URLs as JSON
- fails fast with a timeout if the browser-backed extraction stalls

Operational note:
- when imported by `property-assessor`, Zillow photo extraction is allowed a longer source-specific timeout than the generic helper default, because some exact-unit Zillow listings expose the correct photo set only after a slower render path

Expected success shape:
- `complete: true`
- `expectedPhotoCount` matches `photoCount`
- `imageUrls` contains the listing photo set

### Zillow identifiers

```bash
cd ~/.openclaw/workspace/skills/web-automation/scripts
node zillow-identifiers.js "https://www.zillow.com/homedetails/6702-Everhart-Rd-APT-T106-Corpus-Christi-TX-78413/2067445642_zpid/"
```

What it does:
- opens the Zillow listing shell without forcing the photo workflow
- inspects embedded `__NEXT_DATA__` plus visible listing text
- extracts parcel/APN-style identifiers when Zillow exposes them
- returns those identifiers so `property-assessor` can use them as stronger CAD lookup keys than listing geo IDs

### HAR

```bash
cd ~/.openclaw/workspace/skills/web-automation/scripts
node har-photos.js "https://www.har.com/homedetail/4141-whiteley-dr-corpus-christi-tx-78418/14069438"
```

What it does:
- opens the HAR listing page
- clicks `Show all photos` / `View all photos`
- extracts the direct `pics.harstatic.com` image URLs from the all-photos page
- fails fast with a timeout if the browser-backed extraction stalls

Expected success shape:
- `complete: true`
- `expectedPhotoCount` matches `photoCount`
- `imageUrls` contains the listing photo set

### Test commands

From `skills/web-automation/scripts`:

```bash
node check-install.js
npm run test:photos
node zillow-discover.js "<street-address>"
node har-discover.js "<street-address>"
node zillow-photos.js "<zillow-listing-url>"
node har-photos.js "<har-listing-url>"
```

Use the live Zillow and HAR URLs above for a known-good regression check.

## One-shot extraction (`extract.js`)

Use `extract.js` when the task is just: open one URL, render it, and return structured content.

### Features

- JavaScript rendering
- lightweight stealth and bounded anti-bot shaping
- JSON-only output
- optional screenshot and saved HTML
- browser sandbox left enabled

### Options

```bash
WAIT_TIME=5000 node skills/web-automation/scripts/extract.js "https://example.com"
SCREENSHOT_PATH=/tmp/page.png node skills/web-automation/scripts/extract.js "https://example.com"
SAVE_HTML=true node skills/web-automation/scripts/extract.js "https://example.com"
HEADLESS=false node skills/web-automation/scripts/extract.js "https://example.com"
USER_AGENT="Mozilla/5.0 ..." node skills/web-automation/scripts/extract.js "https://example.com"
```

### Output fields

- `requestedUrl`
- `finalUrl`
- `title`
- `content`
- `metaDescription`
- `status`
- `elapsedSeconds`
- `challengeDetected`
- optional `screenshot`
- optional `htmlFile`

## Persistent browsing profile

`browse.ts`, `auth.ts`, `flow.ts`, and `scrape.ts` use a persistent CloakBrowser profile so sessions survive across runs.

Canonical env vars:

- `CLOAKBROWSER_PROFILE_PATH`
- `CLOAKBROWSER_HEADLESS`
- `CLOAKBROWSER_USERNAME`
- `CLOAKBROWSER_PASSWORD`

Legacy aliases still supported for compatibility:

- `CAMOUFOX_PROFILE_PATH`
- `CAMOUFOX_HEADLESS`
- `CAMOUFOX_USERNAME`
- `CAMOUFOX_PASSWORD`

## Natural-language flow runner (`flow.ts`)

Use `flow.ts` when you want a general command style like:

- "go to this site"
- "find this button and click it"
- "type this and press enter"

### Example

```bash
npx tsx flow.ts --instruction 'go to https://example.com then click on "Sign in" then type "stef@example.com" in #email then press enter'
```

You can also use JSON steps for deterministic runs:

```bash
npx tsx flow.ts --steps '[{"action":"goto","url":"https://example.com"},{"action":"click","text":"Sign in"}]'
```
