158 Commits

Author SHA1 Message Date
stefano e523b34d1b fix: codex uses --yolo not --full-auto 2026-05-18 19:15:59 -05:00
stefano fd1d2c3e92 fix: invoke all CLI clients in full-access/yolo mode
- codex: --full-auto
- claude: --dangerously-skip-permissions
- opencode: --dangerously-skip-permissions
2026-05-18 19:15:04 -05:00
stefano d3aa92be0d fix: use 'opencode run' instead of bare prompt for OpenCode client 2026-05-18 19:06:34 -05:00
stefano 0e273b59ec chore: add ai_plan/ to .gitignore 2026-05-18 18:51:32 -05:00
stefano 2e884e49c8 merge M6 into implement/2026-05-18-stef-openclaw-skills 2026-05-18 18:44:07 -05:00
stefano 775a665eaa feat(M6): Documentation 2026-05-18 18:44:07 -05:00
stefano 2103c424f4 merge S-604 into M6 2026-05-18 18:43:58 -05:00
stefano 32f8a23700 merge S-603 into M6 2026-05-18 18:43:58 -05:00
stefano 480958f12e feat(S-603): Create docs/architecture.md 2026-05-18 18:43:58 -05:00
stefano c188f09684 feat(S-604): Update README.md and docs/README.md 2026-05-18 18:43:08 -05:00
stefano 7818e78244 merge S-602 into M6 2026-05-18 18:41:58 -05:00
stefano c35ffe8af5 merge S-601 into M6 2026-05-18 18:41:58 -05:00
stefano a6f855c9d9 feat(S-602): Create docs/installation.md 2026-05-18 18:41:58 -05:00
stefano 52675f6dc1 feat(S-601): Create docs/ai-cli-dispatch.md 2026-05-18 18:41:58 -05:00
stefano d87038204b merge M5 into implement/2026-05-18-stef-openclaw-skills 2026-05-18 18:39:33 -05:00
stefano 4f59258b20 feat(M5): CLI Integration 2026-05-18 18:39:33 -05:00
stefano 0879ffe39f merge M4 into implement/2026-05-18-stef-openclaw-skills 2026-05-18 18:14:48 -05:00
stefano fe7a015ca4 feat(M4): Natural Language Dispatch 2026-05-18 18:14:48 -05:00
stefano 0c9248d5ca merge S-401 into M4 2026-05-18 18:14:13 -05:00
stefano 7fa959d115 feat(S-401): Test-drive and implement src/dispatch.ts 2026-05-18 18:14:13 -05:00
stefano 50443373bd merge M3 into implement/2026-05-18-stef-openclaw-skills 2026-05-18 18:11:45 -05:00
stefano a2cfa7027e feat(M3): Direct Execution 2026-05-18 18:11:45 -05:00
stefano fe94629797 merge S-301 into M3 2026-05-18 18:01:51 -05:00
stefano a99041f910 feat(S-301): Test-drive and implement src/execute.ts 2026-05-18 18:01:51 -05:00
stefano 360e27d952 merge M2 into implement/2026-05-18-stef-openclaw-skills 2026-05-18 17:53:47 -05:00
stefano 82fcd3363c feat(M2): Client Detection & Configuration 2026-05-18 17:53:47 -05:00
stefano 185083ace8 merge S-203 into M2 2026-05-18 17:51:44 -05:00
stefano 167cdb6ffe merge S-202 into M2 2026-05-18 17:51:44 -05:00
stefano f3458734d4 feat(S-203): Test-drive and implement src/config.ts 2026-05-18 17:51:44 -05:00
stefano 2642c280a2 feat(S-202): Test-drive and implement src/detect.ts 2026-05-18 17:51:07 -05:00
stefano 8340933f8a merge M1 into implement/2026-05-18-stef-openclaw-skills 2026-05-18 17:45:27 -05:00
stefano 4629fe17de feat(M1): Project Scaffold 2026-05-18 17:45:27 -05:00
stefano 949bd05420 merge S-103 into M1 2026-05-18 17:40:10 -05:00
stefano 162517c0e0 feat(S-103): Create scripts/ai-cli-dispatch launcher 2026-05-18 17:40:10 -05:00
stefano 4a6cacb21d merge S-201 into M1 2026-05-18 17:38:35 -05:00
stefano 47f555a367 feat(S-201): Create src/types.ts with shared type definitions 2026-05-18 17:38:35 -05:00
stefano 445d9bfdee merge S-102 into M1 2026-05-18 17:36:31 -05:00
stefano 28e6bbba74 merge S-101 into M1 2026-05-18 17:36:31 -05:00
stefano 50928313a1 feat(S-102): Create package.json, tsconfig.json, .gitignore 2026-05-18 17:36:31 -05:00
stefano fb01334273 feat(S-101): Create SKILL.md with YAML frontmatter 2026-05-18 17:36:21 -05:00
stefano b3875858c7 fix(amazon-shopping): enforce rating filters in chat output 2026-04-15 21:05:27 -05:00
stefano fda0602ac9 fix(amazon-shopping): verify prime and delivery filters 2026-04-15 20:28:16 -05:00
stefano a81a055ec6 docs(amazon-shopping): add install and update instructions 2026-04-15 19:19:09 -05:00
stefano 4204d28077 fix(amazon-shopping): harden agent invocation 2026-04-15 19:09:52 -05:00
stefano c1286e9c42 docs(amazon-shopping): document amazon product search skill 2026-04-15 18:51:42 -05:00
stefano 1e0e265f1e feat(amazon-shopping): scrape and filter amazon product results 2026-04-15 18:48:51 -05:00
stefano ef326896f4 feat(amazon-shopping): parse filters and extract search candidates 2026-04-15 18:31:44 -05:00
stefano 8ad532545d feat(amazon-shopping): scaffold amazon product search skill 2026-04-15 18:24:13 -05:00
stefano 26a968797c Fix spotify m3u Windows path parsing 2026-04-12 10:03:43 -05:00
stefano c2db2b51e7 merge: spotify skill implementation 2026-04-12 02:17:32 -05:00
stefano eb66d96ef3 docs(spotify): implement milestone M5 install docs 2026-04-12 02:09:51 -05:00
stefano 141488c0f2 feat(spotify): implement milestone M4 importers 2026-04-12 02:00:50 -05:00
stefano d8570edcf0 feat(spotify): implement milestone M3 api commands 2026-04-12 01:52:18 -05:00
stefano c8c0876b7c feat(spotify): implement milestone M2 auth 2026-04-12 01:36:27 -05:00
stefano f7dfb7d71d feat(spotify): implement milestone M1 scaffold 2026-04-12 01:28:47 -05:00
stefano a91b82ae32 chore(skills): remove elevenlabs stt skill 2026-04-11 23:31:49 -05:00
stefano b9878e938c Clarify flight-finder cron completion rules 2026-04-01 10:21:14 -05:00
stefano b2e97a3036 Polish flight-finder PDF layout 2026-03-30 19:54:35 -05:00
stefano 809a3955e5 Improve flight-finder report links and cron workflow 2026-03-30 18:26:09 -05:00
stefano fb868b9e5f fix(flight-finder): require fresh search evidence and improve PDF layout 2026-03-30 17:45:32 -05:00
stefano e2657f4850 feat(flight-finder): implement milestone M3 - prompt migration and smoke test fixtures 2026-03-30 17:12:52 -05:00
stefano c30ad85e0d feat(flight-finder): implement milestone M2 - report workflow and delivery gates 2026-03-30 17:00:09 -05:00
stefano ba5b0e4e67 chore(flight-finder): stop tracking installed dependencies 2026-03-30 16:46:06 -05:00
stefano 9c7103770a feat(flight-finder): implement milestone M1 - domain model and skill contract 2026-03-30 16:45:40 -05:00
stefano 57f6b132b2 Fix NordVPN DNS and Tailscale recovery interlock 2026-03-30 14:29:39 -05:00
stefano b3a59b5b45 fix(nordvpn-client): validate live utun persistence before dns pinning 2026-03-30 12:08:25 -05:00
stefano a796481875 feat(nordvpn-client): gate macos connect on stable wireguard persistence 2026-03-30 11:55:20 -05:00
stefano 8d2c162849 feat(nordvpn-client): implement milestone M1 diagnostics and classification 2026-03-30 11:34:20 -05:00
stefano 4919edcec1 Fix ACP startup guidance for managed acpx path 2026-03-30 08:05:53 -05:00
stefano 9f3d080471 Document ACP rollout caveats and parity results 2026-03-30 00:02:04 -05:00
stefano efbdb25937 Document ACP orchestration baseline 2026-03-29 23:35:18 -05:00
stefano b77134ced5 Use Zillow parcel hints for CAD lookup 2026-03-28 03:55:56 -05:00
stefano ece8fc548f Trust embedded Zillow photo sets without visible count 2026-03-28 03:40:50 -05:00
stefano 446d43cc78 Prefer structured Zillow photo data before click path 2026-03-28 03:30:42 -05:00
stefano 54854edfc6 Harden assessor fallback after Zillow photo failure 2026-03-28 03:17:51 -05:00
stefano 3335e96d35 Refresh repo docs for assessor and integrations 2026-03-28 02:55:54 -05:00
stefano a7c318aca8 Route property-assessor email through Luke 2026-03-28 02:53:34 -05:00
stefano 9caa8fa4f5 Align google-workspace helper docs 2026-03-28 02:51:32 -05:00
stefano c3e5f669ed Fix Stefano PDF email delivery path 2026-03-28 02:50:42 -05:00
stefano 5cffd0edf9 Prefer web-automation over brave for property assessor chat runs 2026-03-28 02:38:10 -05:00
stefano 8fe451e8d0 Fix slower Zillow unit photo discovery path 2026-03-28 02:28:30 -05:00
stefano 7690dc259b Enrich property assessor with CAD detail data 2026-03-28 02:07:18 -05:00
stefano b1722a04fa fix: require completed photo review before pdf render 2026-03-28 01:43:34 -05:00
stefano a88d960ec9 docs: treat explicit email target as send authorization 2026-03-28 01:40:47 -05:00
stefano d2d43df24d docs: forbid silent helper polling in whatsapp runs 2026-03-28 01:34:23 -05:00
stefano 3d7ce7617c fix: make property-assessor safer for whatsapp runs 2026-03-28 01:28:59 -05:00
stefano 2deeb31369 Clarify chat-safe property assessor behavior 2026-03-28 01:05:00 -05:00
stefano 33466079a3 Require property assessor follow-through after preliminary helper 2026-03-28 00:44:28 -05:00
stefano be4f829704 Require explicit property assessor purpose per request 2026-03-28 00:40:35 -05:00
stefano 761bd2f083 Block preliminary property assessor PDFs 2026-03-28 00:34:28 -05:00
stefano c68523386d Fix property assessor geocode fallback 2026-03-28 00:25:31 -05:00
stefano 7570f748f0 Clarify approval-safe property assessor flow 2026-03-27 23:47:24 -05:00
stefano 1f23eac52c Defer property assessor email gate 2026-03-27 23:27:51 -05:00
stefano f8c998d579 Make listing discovery unit-aware 2026-03-27 23:11:10 -05:00
stefano 301986fb25 Add purpose-aware property assessor intake 2026-03-27 23:01:12 -05:00
stefano c58a2a43c8 Add property assessor assess command 2026-03-27 22:35:57 -05:00
stefano e6d987d725 Port property assessor helpers to TypeScript 2026-03-27 22:23:58 -05:00
stefano 954374ce48 Expand property assessor documentation 2026-03-27 21:39:06 -05:00
stefano 19adb919fc Refresh property assessor and web automation docs 2026-03-27 21:35:55 -05:00
stefano eeea0c8ef1 Add Zillow and HAR photo extractors 2026-03-27 17:35:46 -05:00
stefano e7c56fe760 Keep Zillow photo workflow inside web-automation 2026-03-27 16:17:45 -05:00
stefano 35d3fede49 Prefer Zillow scroller image URLs for photo review 2026-03-27 16:11:56 -05:00
stefano f82980c4ed Require Zillow photo review via web-automation all-photos flow 2026-03-27 14:56:28 -05:00
stefano 2c5d31f85e Require action-based photo-source attempts in property-assessor 2026-03-27 14:50:14 -05:00
stefano 02cc5f8e7e Require photo-source fallback attempts in property-assessor 2026-03-27 14:43:58 -05:00
stefano 8ec0237309 Require explicit photo-review status in property assessments 2026-03-27 14:06:14 -05:00
stefano 93247a5954 Make property-assessor approval-averse for chat use 2026-03-27 13:38:53 -05:00
stefano c2ae79ccf0 Prefer accessible all-photos views for property photo review 2026-03-27 13:15:06 -05:00
stefano 90635bf8f2 Improve gallery photo-review guidance for property assessment 2026-03-27 12:02:47 -05:00
stefano b13f272e48 feat: add photo and make-ready analysis to property assessor 2026-03-27 10:36:42 -05:00
stefano 075f5bd9a7 feat: add property assessor skill and web automation approvals docs 2026-03-27 10:16:17 -05:00
Stefano Fiorini b2bb07fa90 feat: make us-cpa questions retrieval-first 2026-03-15 04:40:57 -05:00
Stefano Fiorini b4f9666560 fix: support surviving spouse filing status in us-cpa 2026-03-15 04:24:37 -05:00
Stefano Fiorini f219672a2e fix: infer filing status from us-cpa questions 2026-03-15 04:20:43 -05:00
Stefano Fiorini 12838f7449 docs: preserve us-cpa workspace virtualenv on sync 2026-03-15 04:05:29 -05:00
Stefano Fiorini 31ed267027 docs: use python3 pip bootstrap for us-cpa install 2026-03-15 04:01:07 -05:00
Stefano Fiorini 850e89d339 docs: require pip upgrade for us-cpa editable install 2026-03-15 03:54:43 -05:00
Stefano Fiorini b520bdc998 docs: quote us-cpa extras install command 2026-03-15 03:53:15 -05:00
stefano 7187ba9ea3 Merge pull request 'us-cpa: OpenClaw skill wrapper for U.S. federal individual tax work' (#1) from feat/us-cpa into main
Reviewed-on: #1
2026-03-15 08:48:24 +00:00
Stefano Fiorini fdfc9f0996 docs: use home-relative us-cpa install paths 2026-03-15 03:35:24 -05:00
Stefano Fiorini 9f650faf88 docs: add us-cpa openclaw installation guide 2026-03-15 03:31:52 -05:00
Stefano Fiorini 1be0317192 test: add us-cpa module coverage and citations 2026-03-15 03:11:24 -05:00
Stefano Fiorini fb39fe76cb fix: expand us-cpa extraction review and rendering 2026-03-15 03:01:16 -05:00
Stefano Fiorini 6c02e0b7c6 fix: add us-cpa tax year rules and package metadata 2026-03-15 02:47:14 -05:00
Stefano Fiorini d3fd874330 docs: finalize us-cpa integration and fixtures 2026-03-15 01:34:14 -05:00
Stefano Fiorini 10a9d40f1d feat: add us-cpa review workflow 2026-03-15 01:31:43 -05:00
Stefano Fiorini 82cf3d9010 feat: add us-cpa preparation workflow 2026-03-15 01:28:22 -05:00
Stefano Fiorini decf3132d5 feat: add us-cpa pdf renderer 2026-03-15 01:26:29 -05:00
Stefano Fiorini c3c0d85908 feat: add us-cpa return model and calculations 2026-03-15 01:23:47 -05:00
Stefano Fiorini 8f797b3a51 feat: add us-cpa question engine 2026-03-15 01:17:14 -05:00
Stefano Fiorini faff555757 feat: add us-cpa case intake workflow 2026-03-15 00:56:07 -05:00
Stefano Fiorini 0c2e34f2f0 feat: add us-cpa tax-year source corpus 2026-03-15 00:53:18 -05:00
Stefano Fiorini 291b729894 feat: scaffold us-cpa skill 2026-03-15 00:45:06 -05:00
Stefano Fiorini 59dbaf8a6c fix: relax CloakBrowser prerequisite check 2026-03-13 09:33:53 -05:00
Stefano Fiorini 8d5dd046a4 fix: harden mac nordvpn status inference 2026-03-13 00:30:06 -05:00
Stefano Fiorini fc8e388c0a chore: update cloakbrowser web automation runtime 2026-03-13 00:19:54 -05:00
Stefano Fiorini e6dccb5656 docs: expand nordvpn client setup and troubleshooting 2026-03-12 02:45:21 -05:00
Stefano Fiorini 7d8eb89911 fix: redact nordvpn path metadata by default 2026-03-12 02:37:16 -05:00
Stefano Fiorini 60f425a4fc fix: stabilize mac nordvpn state reporting 2026-03-12 02:28:57 -05:00
Stefano Fiorini 647828aa78 fix: force mac nordvpn disconnect teardown 2026-03-12 02:22:50 -05:00
Stefano Fiorini b4c8d3fdb8 fix: clear stale nordvpn disconnect state 2026-03-12 02:08:34 -05:00
Stefano Fiorini 09b1c1e37a fix: simplify mac nordvpn tailscale coordination 2026-03-12 01:55:46 -05:00
Stefano Fiorini 916d8bf95a docs: add nordvpn tailscale coordination plan 2026-03-12 01:46:52 -05:00
Stefano Fiorini 6bc21219a7 docs: add nordvpn macos dns plan 2026-03-12 01:35:32 -05:00
Stefano Fiorini ca33b2d74a fix: avoid mac wireguard dns rewrites 2026-03-12 01:20:02 -05:00
Stefano Fiorini d0c50f5d8a fix: harden nordvpn wireguard verification 2026-03-12 00:50:05 -05:00
Stefano Fiorini a8a285b356 feat: add nordvpn wireguard sudo helper 2026-03-12 00:34:04 -05:00
Stefano Fiorini 045cf6aad2 feat: add default nordvpn credential paths 2026-03-12 00:09:56 -05:00
Stefano Fiorini 78be4fc600 docs: clarify nordvpn mac setup flow 2026-03-11 23:53:15 -05:00
Stefano Fiorini d1b4d58c5d Merge branch 'feature/nordvpn-wireguard' 2026-03-11 23:44:26 -05:00
Stefano Fiorini 4a539a33c9 feat: add mac wireguard nordvpn backend 2026-03-11 23:44:22 -05:00
Stefano Fiorini b326153d26 fix: clarify mac nordvpn app-only mode 2026-03-11 22:59:08 -05:00
Stefano Fiorini 2612cef1dc Merge branch 'feature/nordvpn-client' 2026-03-11 22:36:03 -05:00
Stefano Fiorini 120721bbc6 feat: add nordvpn client skill 2026-03-11 22:35:50 -05:00
Stefano Fiorini fe5b4659fe docs: add nordvpn client plan 2026-03-11 22:02:49 -05:00
Stefano Fiorini 4078073f0b refactor: migrate web-automation to cloakbrowser 2026-03-11 20:26:59 -05:00
Stefano Fiorini c9254ed7eb docs: add cloakbrowser migration plan 2026-03-11 20:14:40 -05:00
Stefano Fiorini 49d0236a52 docs: add web automation consolidation plans 2026-03-10 19:24:59 -05:00
227 changed files with 28385 additions and 809 deletions
+2
View File
@@ -1 +1,3 @@
.worktrees/
node_modules/
ai_plan/
+22 -3
View File
@@ -11,23 +11,42 @@ This repository contains practical OpenClaw skills and companion integrations. I
- Per-skill runtime instructions: `skills/<skill-name>/SKILL.md`
- Integration implementation files: `integrations/<integration-name>/`
- Integration docs: `docs/*.md`
- Tool implementation files: `tools/<tool-name>/`
- Tool docs: `docs/*.md`
## Skills
| Skill | What it does | Path |
|---|---|---|
| `elevenlabs-stt` | Transcribe local audio files with ElevenLabs Speech-to-Text, with diarization, language hints, event tags, and JSON output. | `skills/elevenlabs-stt` |
| `amazon-shopping` | Search Amazon.com product results with bounded web-automation, price/unit-price/width/Prime/delivery filters, specs, ratings, review metadata, and chat-safe result blocks with direct links. | `skills/amazon-shopping` |
| `flight-finder` | Collect structured flight-search inputs, run bounded multi-source travel searches, rank results in USD, and gate fixed-template PDF/email delivery through Lukes sender path. | `skills/flight-finder` |
| `gitea-api` | Interact with Gitea via REST API (repos, issues, PRs, releases, branches, user info). | `skills/gitea-api` |
| `nordvpn-client` | Install, log in to, connect, disconnect, and verify NordVPN sessions across Linux CLI and macOS NordLynx/WireGuard backends. | `skills/nordvpn-client` |
| `portainer` | Manage Portainer stacks via API (list, start/stop/restart, update, prune images). | `skills/portainer` |
| `property-assessor` | Assess a residential property from an address or listing URL with CAD/public-record enrichment, Zillow-first/HAR-fallback photo review, carry-cost/risk analysis, and fixed-template PDF output. | `skills/property-assessor` |
| `searxng` | Search through a local or self-hosted SearXNG instance for web, news, images, and more. | `skills/searxng` |
| `web-automation` | One-shot extraction plus broader browsing/scraping with Playwright + Camoufox (auth flows, extraction, bot-protected sites). | `skills/web-automation` |
| `spotify` | Search Spotify tracks, manage playlists, and import songs from text files, M3U playlists, or music folders through the Spotify Web API. | `skills/spotify` |
| `us-cpa` | Federal individual 1040 workflow for tax questions, case intake, preparation, review, and draft e-file-ready export. | `skills/us-cpa` |
| `web-automation` | One-shot extraction plus broader browsing/scraping with CloakBrowser, including unit-aware Zillow/HAR discovery and dedicated listing-photo extractors. | `skills/web-automation` |
## Integrations
| Integration | What it does | Path |
|---|---|---|
| `google-maps` | Traffic-aware ETA and leave-by calculations using Google Maps APIs. | `integrations/google-maps` |
| `google-workspace` | Gmail and Google Calendar helper CLI for profile, mail, calendar search, and event creation. | `integrations/google-workspace` |
| `google-workspace` | Gmail and Google Calendar helper CLI for profile, mail, attachment-capable send, calendar search, and event creation. | `integrations/google-workspace` |
## Tools
| Tool | What it does | Path |
|---|---|---|
| `ai-cli-dispatch` | Dispatch AI CLI coding tasks to available clients (Codex, Claude Code, OpenCode) with automatic discovery, version checking, and execution. | `tools/ai-cli-dispatch` |
## Operator docs
| Doc | What it covers |
|---|---|
| `docs/openclaw-acp-orchestration.md` | ACP orchestration setup for Codex and Claude Code on the gateway host |
## Install ideas
+18 -3
View File
@@ -4,13 +4,28 @@ This folder contains detailed docs for each skill in this repository.
## Skills
- [`elevenlabs-stt`](elevenlabs-stt.md) — Local audio transcription through ElevenLabs Speech-to-Text
- [`amazon-shopping`](amazon-shopping.md) — Amazon.com product search with local web-automation, product filters, pricing, delivery, specs, and review metadata
- [`flight-finder`](flight-finder.md) — Typed flight-search intake, bounded source orchestration, PDF report rendering, and Luke-sender email delivery
- [`gitea-api`](gitea-api.md) — REST-based Gitea automation (no `tea` CLI required)
- [`nordvpn-client`](nordvpn-client.md) — Cross-platform NordVPN install, login, connect, disconnect, and verification with Linux CLI and macOS NordLynx/WireGuard support
- [`portainer`](portainer.md) — Portainer stack management (list, lifecycle, updates, image pruning)
- [`property-assessor`](property-assessor.md) — Residential property assessment with CAD/public-record enrichment, Zillow/HAR photo review, valuation workflow, and PDF delivery rules
- [`searxng`](searxng.md) — Privacy-respecting metasearch via a local or self-hosted SearXNG instance
- [`web-automation`](web-automation.md) — One-shot extraction plus Playwright + Camoufox browser automation and scraping
- [`spotify`](spotify.md) — Spotify Web API helper for track search, playlist management, and text/M3U/folder imports
- [`us-cpa`](us-cpa.md) — Federal individual 1040 workflow for tax questions, case intake, preparation, review, and draft e-file-ready export
- [`web-automation`](web-automation.md) — One-shot extraction plus CloakBrowser automation, including unit-aware Zillow/HAR discovery and dedicated photo extraction
## Integrations
- [`google-maps`](google-maps.md) — Traffic-aware ETA and leave-by calculations via Google Maps APIs
- [`google-workspace`](google-workspace.md) — Gmail and Google Calendar helper CLI
- [`google-workspace`](google-workspace.md) — Gmail and Google Calendar helper CLI with attachment-capable send support
## Tools
- [`ai-cli-dispatch`](ai-cli-dispatch.md) — Dispatch AI CLI coding tasks to available clients with automatic discovery, version checking, and execution
- [`installation`](installation.md) — Prerequisites, install steps, PATH configuration, and optional config file setup for `ai-cli-dispatch`
- [`architecture`](architecture.md) — Design decisions, module breakdown, data flow, coexistence with ACP, and extension points for `ai-cli-dispatch`
## Operator Docs
- [`openclaw-acp-orchestration`](openclaw-acp-orchestration.md) — OpenClaw ACP orchestration for Codex and Claude Code on the gateway host
+258
View File
@@ -0,0 +1,258 @@
# ai-cli-dispatch
Dispatch AI CLI coding tasks to available clients (Codex, Claude Code, OpenCode) with automatic discovery, version checking, and execution.
## Scope
- discover installed AI CLI clients on the host
- report client versions and availability
- dispatch a prompt to a specific client by name
- auto-resolve the best client from prompt keywords
- forward arguments natively to each client
The tool is a lightweight sync-only dispatcher. It does not implement streaming, chat sessions, or ACP orchestration. For ACP-based harnesses, see `docs/openclaw-acp-orchestration.md`.
## Setup
From the repo or installed skill directory:
```bash
cd tools/ai-cli-dispatch
npm install
```
The dispatcher itself requires only Node.js 20+ and `npm`. The actual AI CLI clients (`codex`, `claude`, `opencode`) are discovered from the host `PATH`; they are not bundled.
## Commands
```bash
ai-cli-dispatch list [--json|--text]
ai-cli-dispatch run --client <client> --prompt <prompt> [--json|--text]
ai-cli-dispatch dispatch <prompt> [--client <client>] [--json|--text]
ai-cli-dispatch --help
```
### `list`
Discover and report all supported clients.
```bash
ai-cli-dispatch list --json
```
Example JSON output:
```json
[
{
"name": "codex",
"path": "/usr/local/bin/codex",
"version": "1.2.3",
"found": true
},
{
"name": "claude",
"found": false
},
{
"name": "opencode",
"path": "/opt/homebrew/bin/opencode",
"version": "0.5.1",
"found": true
}
]
```
Use `--text` for human-readable checkmarks:
```bash
ai-cli-dispatch list --text
```
### `run`
Execute a prompt directly through a named client.
```bash
ai-cli-dispatch run --client codex --prompt "refactor this function"
ai-cli-dispatch run --client claude --prompt "add tests for auth middleware"
ai-cli-dispatch run --client opencode --prompt "migrate to ESM"
```
The prompt is forwarded with each clients native argument shape:
| Client | Arguments passed |
|---|---|
| `codex` | `exec "<prompt>"` |
| `claude` | `-p "<prompt>"` |
| `opencode` | `"<prompt>"` |
### `dispatch`
Auto-resolve the client from prompt keywords, then execute.
```bash
ai-cli-dispatch dispatch "use claude to write tests"
ai-cli-dispatch dispatch "codex refactor auth module"
ai-cli-dispatch dispatch "opencode migrate to ESM"
```
Keyword matching is case-insensitive and ordered:
1. `--client` flag (highest precedence)
2. `"open code"` (spaced variant) → `opencode`
3. `"claude"``claude`
4. `"codex"``codex`
5. `"opencode"``opencode`
6. `defaultClient` from config (lowest precedence)
Override auto-resolution explicitly:
```bash
ai-cli-dispatch dispatch "fix the bug" --client claude
```
## Client Discovery
Discovery searches `PATH` in this order for each client name:
1. `codex` — OpenAI Codex CLI
2. `claude` — Anthropic Claude Code
3. `opencode` — OpenCode CLI
The search uses `which` (or `where` on Windows) first, then falls back to a manual `PATH` directory scan. If a binary is found, `--version` is invoked to extract a semver string.
## Configuration
Optional config file:
```text
~/.openclaw/ai-cli-dispatch.json
```
Example:
```json
{
"paths": {
"codex": "/usr/local/bin/codex",
"claude": "/opt/homebrew/bin/claude"
},
"defaultClient": "claude"
}
```
Resolution priority for paths and default client (highest to lowest):
1. CLI flag (`--client`, `--codex-path`, etc.)
2. Environment variable (`AI_CLI_CODEX_PATH`, `AI_CLI_DEFAULT_CLIENT`, etc.)
3. Config file (`paths`, `defaultClient`)
4. `which` / `where` discovery
Supported env vars:
| Variable | Purpose |
|---|---|
| `AI_CLI_CODEX_PATH` | Override `codex` binary path |
| `AI_CLI_CLAUDE_PATH` | Override `claude` binary path |
| `AI_CLI_OPENCODE_PATH` | Override `opencode` binary path |
| `AI_CLI_DEFAULT_CLIENT` | Override default client (`codex`, `claude`, or `opencode`) |
## Output Model
Default output is JSON. Use `--text` to stream raw `stdout`/`stderr` directly.
JSON success shape (`run` and `dispatch`):
```json
{
"stdout": "...",
"stderr": "...",
"exitCode": 0
}
```
JSON error shape:
```json
{
"error": "..."
}
```
Exit codes:
| Code | Meaning |
|---|---|
| `0` | Success |
| `1` | Missing/unknown command, missing argument, unknown client, resolution failure, or execution error |
## Error Handling Guidance
### `Client "<name>" not found or not installed`
Meaning: the requested client binary is not on `PATH` and not overridden by config.
Actions:
1. Confirm the client is installed (`codex --version`, `claude --version`, etc.)
2. Check that its directory is on `PATH`
3. Or override the path in `~/.openclaw/ai-cli-dispatch.json`
### `Prompt cannot be empty`
Meaning: the prompt string was empty or whitespace-only.
Action: supply a non-empty `--prompt` or positional prompt argument.
### `Execution timed out after 300000ms`
Meaning: the client subprocess did not finish within the default 5-minute timeout.
Action: the client may be waiting for interactive input or the task is too large. Break the prompt into smaller pieces, or run the client directly to diagnose.
### `Could not resolve client from prompt`
Meaning: `dispatch` found no matching keyword and no `defaultClient` is configured.
Action: include a client name in the prompt (e.g., `"use claude to ..."`) or set `defaultClient` in config.
## Common Flows
### Check what is installed
```bash
ai-cli-dispatch list --json
```
### Run a quick task through a specific client
```bash
ai-cli-dispatch run --client codex --prompt "fix lint errors in src/app.ts"
```
### Let the tool pick the client from the prompt
```bash
ai-cli-dispatch dispatch "claude: add unit tests for utils.ts"
```
### Force a client when the prompt is ambiguous
```bash
ai-cli-dispatch dispatch "review this PR" --client claude
```
## Coexistence with ACP
`ai-cli-dispatch` is a direct subprocess dispatcher. It runs the client binary synchronously and returns its output. It is not an ACP agent and does not participate in ACP orchestration.
- Use `ai-cli-dispatch` when you need a quick, local, one-shot CLI execution.
- Use ACP (`docs/openclaw-acp-orchestration.md`) when you need session-bound coding harnesses with thread context, multi-turn review, or orchestrator-managed verification gates.
## Implementation Notes
- The dispatcher is TypeScript/Node.js with a single external dependency (`minimist`).
- Client arguments are hardcoded per tool to match each clients stable CLI contract.
- The default timeout is 5 minutes (`300_000` ms).
- On Windows, discovery uses `where` instead of `which` and `.exe` extensions are assumed.
+146
View File
@@ -0,0 +1,146 @@
# Amazon Shopping Skill
`amazon-shopping` searches Amazon.com product results with bounded, read-only web automation and deterministic local filtering.
## Example Invocation
```text
use amazon-shopping to search for 100w led bulbs that cost less than $4 each and have over 200 reviews with a review score of more than 4.5 stars
```
## Helper Commands
Run from the installed skill:
```bash
cd ~/.openclaw/workspace/skills/amazon-shopping
scripts/search-products 'USB-C charger under $30' --limit 10 --json
scripts/search-products '100w led bulbs that cost less than $4 each and have over 200 reviews with a review score of more than 4.5 stars' --limit 5 --markdown
scripts/search-products 'USB-C cable with over 1000 reviews and rating over 4 stars' --limit 3 --json --skip-details
scripts/search-products 'sofa bed of 77 inches or wider, 4 stars or higher, 200+ reviews, shipped with Prime, delivery by tomorrow, top 10 by price' --limit 10 --json --markdown
```
Use `--dry-run` to parse a request and show planned filters without navigating to Amazon:
```bash
scripts/search-products 'USB-C charger under $30' --dry-run --json
```
Use single quotes when a request contains dollar amounts so the shell does not expand `$4`. `--max N` is accepted as a compatibility alias for `--limit N`.
## Install Or Update
Run these commands from the `stef-openclaw-skills` repo checkout:
```bash
cd ~/.openclaw/workspace/projects/stef-openclaw-skills
git pull --ff-only
rsync -a --delete \
--exclude node_modules \
--exclude dist \
--exclude coverage \
--exclude tmp \
--exclude out \
--exclude '*.log' \
--exclude '*.real.html' \
skills/amazon-shopping/ \
~/.openclaw/workspace/skills/amazon-shopping/
cd ~/.openclaw/workspace/skills/amazon-shopping
npm ci
```
Use the same sequence for a first install and for updates. `rsync --delete` keeps the active skill identical to the repo copy while preserving generated dependencies through the `node_modules` exclude. Always run `npm ci` after syncing because dependency changes are tracked through `package-lock.json`.
## Setup Verification
Verify the dependency skill, the active install, and OpenClaw discovery:
```bash
openclaw skills info web-automation
node "$HOME/.openclaw/workspace/skills/web-automation/scripts/check-install.js"
cd ~/.openclaw/workspace/skills/amazon-shopping
npm run lint
npm run typecheck
npm test
openclaw skills info amazon-shopping
```
For a quick parser-only check that does not browse Amazon:
```bash
scripts/search-products 'USB-C charger under $30' --dry-run --json
```
For a live smoke after install or update:
```bash
scripts/search-products '100w led bulbs that cost less than $4 each and have over 200 reviews with a review score of more than 4.5 stars' --limit 5 --json --markdown
```
## Dependency
This skill depends on the workspace `web-automation` skill and its CloakBrowser runtime.
## Fields
Each result includes the product ASIN, title, source URL, price, unit price when visible, rating, review count, delivery summary, specs, feature bullets, seller, availability, sponsored marker, matched filters, missing fields, and extraction notes.
Unknown or hidden fields stay unknown. The skill does not invent delivery dates, star histograms, prices, or review counts.
Markdown output uses chat-safe result blocks intended for direct user-facing answers in WhatsApp, Telegram, and terminals. Each product must keep a direct URL line:
```markdown
## Best Matches
1. Example Sofa Bed
Price: $399.99
Rating: 4.3 stars
Reviews: 250
Width: 83" OK
Prime: Prime OK
Delivery: FREE delivery Tomorrow OK
Link: https://www.amazon.com/dp/ASIN
```
The `OK` / `NO` marker is only attached to fields that correspond to requested filters. For example, `Prime OK` means the helper found a Prime signal on the search card or detail page; `not verified NO` means the product did not pass a requested Prime filter.
## Filters
Supported request filters include:
- minimum rating
- minimum review count
- maximum product price
- maximum unit price
- minimum width in inches
- Prime delivery
- delivery by today, tomorrow, or overnight
- sort by price
- result limit
- maximum search pages
`over 200 reviews` and `more than 4.5 stars` are strict comparisons. `at least 200 reviews` and `4.5 stars or better` are inclusive comparisons.
Examples of supported natural-language filters:
- `77 inches or wider`
- `shipped with Prime`
- `delivery by tomorrow`
- `overnight shipping`
- `top 10 by price`
- `rating 4.0 or better`
Equivalent CLI flags:
```bash
scripts/search-products 'sofa bed beige' --min-rating 4 --min-reviews 200 --min-width 77 --require-prime --delivery-by tomorrow --sort-by price --limit 10 --markdown
```
## Guardrails
This skill is for operator-directed product research, not purchasing automation.
- It checks Amazon robots directives before live navigation.
- It does not sign in, add to cart, purchase, access wishlists, submit reviews, crawl review pages, or bypass CAPTCHA/block pages.
- It stops and reports a warning when Amazon returns a challenge, block, or disallowed robots path.
- It uses default bounded operation: 15 results, 2 search pages, detail pages one at a time.
+213
View File
@@ -0,0 +1,213 @@
# ai-cli-dispatch Architecture
This document describes the internal design of `ai-cli-dispatch`, the module breakdown, data flow, key design decisions, and how to extend the tool.
## Module Breakdown
```text
src/
├── cli.ts — Entry point: argument parsing, command routing, I/O formatting
├── types.ts — Shared types and error classes
├── constants.ts — Client name registry and platform helpers
├── config.ts — Layered configuration resolution (flags → env → file → PATH)
├── detect.ts — Client discovery: binary lookup and version extraction
├── dispatch.ts — Prompt-to-client resolution (explicit flag → keywords → default)
└── execute.ts — Subprocess spawning, stdout/stderr capture, timeout handling
```
### Responsibilities
| Module | Responsibility |
|---|---|
| `cli.ts` | Parses `argv` with `minimist`, routes to `list` / `run` / `dispatch`, prints JSON or text output, and controls the process exit code. |
| `types.ts` | Defines `ClientName`, `ClientInfo`, `ExecResult`, `ToolConfig`, and the error hierarchy (`ClientNotFoundError`, `ExecError`). |
| `constants.ts` | Holds the canonical `CLIENT_NAMES` array and `isWindows()` helper used by discovery and config. |
| `config.ts` | Resolves per-client binary paths and the optional `defaultClient` from four layered sources. |
| `detect.ts` | Locates each client binary on `PATH`, falls back to a manual directory scan, and invokes `--version` to extract a semver string. |
| `dispatch.ts` | Chooses the target client from a prompt string using ordered keyword matching, with overrides for explicit `--client` and `defaultClient`. |
| `execute.ts` | Spawns the chosen client with its native argument shape, buffers `stdout`/`stderr`, enforces a timeout, and returns an `ExecResult` or throws a typed error. |
## Data Flow
A typical `dispatch` invocation flows through four stages:
```
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ detect │ ──► │ config │ ──► │ dispatch │ ──► │ execute │
└─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘
│ │ │ │
which/where flags/env/file keyword scan spawn child
PATH walk defaultClient --client override capture output
--version fallback default timeout / exitCode
```
### 1. Detect
`detectClients()` iterates over `CLIENT_NAMES` and attempts to locate each binary:
1. Invoke `which <name>` (or `where <name>` on Windows).
2. If that fails, walk `PATH` segments manually and test `existsSync()`.
3. If a binary is found, run `<binary> --version` and parse the first semver-like match.
Result: an array of `ClientInfo` objects with `name`, `found`, `path`, and `version`.
### 2. Config
`resolveConfig()` builds a `ResolvedConfig` by layering sources (highest to lowest precedence):
1. **CLI flags**`--codex-path`, `--claude-path`, `--opencode-path`, `--default-client`
2. **Environment variables**`AI_CLI_CODEX_PATH`, `AI_CLI_CLAUDE_PATH`, `AI_CLI_OPENCODE_PATH`, `AI_CLI_DEFAULT_CLIENT`
3. **Config file**`~/.openclaw/ai-cli-dispatch.json` (`paths` and `defaultClient` keys)
4. **PATH discovery**`which`/`where` fallback via `defaultWhichSync()`
Only values for the three known `ClientName` entries are accepted; unknown `defaultClient` values are ignored.
### 3. Dispatch
`resolveClient(prompt, config)` decides which client to use:
1. If `config.client` is a valid `ClientName`, return it immediately.
2. Lower-case the prompt and scan for substrings in order:
- `"open code"``opencode`
- `"claude"``claude`
- `"codex"``codex`
- `"opencode"``opencode`
3. If no keyword matches, return `config.defaultClient` or `null`.
This ordering intentionally prioritizes `"open code"` before `"opencode"` so the spaced natural-language variant wins.
### 4. Execute
`executePrompt(client, prompt, options)` runs the selected client:
1. Reject empty or whitespace-only prompts with `ExecError`.
2. Validate that an explicit `clientPath` exists on disk (if provided).
3. Map the client to its native argument array via `CLIENT_ARGS`:
- `codex``["exec", prompt]`
- `claude``["-p", prompt]`
- `opencode``[prompt]`
4. `spawn()` the process with `shell: false`.
5. Buffer `stdout` and `stderr` via `"data"` listeners.
6. Start a `setTimeout`; if it fires, `child.kill()` is sent.
7. On `close`, resolve with `{ stdout, stderr, exitCode }`.
8. On `error`, reject with `ClientNotFoundError` for `ENOENT` or `ExecError` for anything else.
9. On timeout, reject with `ExecError` containing the buffered output so far.
The default timeout is **5 minutes** (`300_000` ms).
## Design Decisions
### Coexistence with ACP
`ai-cli-dispatch` is intentionally **not** an ACP agent. It is a thin, local subprocess wrapper with no session state, no thread binding, and no orchestrator protocol.
- Use `ai-cli-dispatch` when you need a quick, one-shot CLI execution on the gateway host.
- Use ACP (`docs/openclaw-acp-orchestration.md`) when you need session-bound coding harnesses, multi-turn review, or orchestrator-managed verification gates.
This separation keeps the dispatcher small and avoids duplicating ACPs scheduling, context persistence, and review-loop responsibilities.
### Keyword Dispatch vs NLP
Client resolution uses deterministic substring matching instead of natural-language parsing or an LLM call.
**Rationale:**
- **Speed:** No network round-trip or model load; resolution is synchronous and sub-millisecond.
- **Predictability:** The same prompt always resolves to the same client. There is no temperature, context window, or model-version drift.
- **Debuggability:** A user can read the ordered keyword list and know exactly why a given prompt resolved to a given client.
- **Scope fit:** The dispatcher only needs to distinguish three clients. A full NLP pipeline would be overkill.
The trade-off is that prompts like `"compare codex and claude"` resolve to `codex` because `"codex"` is checked first. Users can always override with `--client`.
### Sync-Only Initial Release
The current implementation is entirely synchronous from the callers perspective: `executePrompt` returns a promise that resolves only when the child process exits or the timeout fires.
**Rationale:**
- The primary use case is one-shot tasks (refactor, add tests, migrate) where the agent needs the complete output before proceeding.
- Streaming would require a different output contract (callbacks, generators, or an event emitter) and complicates the JSON error model.
- ACP already covers interactive, streaming, and session-based use cases.
Streaming is an intentional future extension point (see below).
### Error Taxonomy
All runtime failures are represented as typed errors so callers and tests can branch precisely:
| Error | When thrown | Data carried |
|---|---|---|
| `ClientNotFoundError` | Binary not on `PATH`, explicit `clientPath` missing, or `ENOENT` from `spawn` | `message` with client name |
| `ExecError` | Empty prompt, unknown client, timeout, non-`ENOENT` spawn error, or child exit | `message` + full `ExecResult` (`stdout`, `stderr`, `exitCode`) |
`ExecError` carries the `ExecResult` so that timeout handlers still return partial output. This avoids losing buffered stdout/stderr when a long-running task is killed.
### Injection-Friendly Module Boundaries
Every non-trivial module accepts an `options` bag with injectable dependencies (`spawnSync`, `spawn`, `existsSync`, `whichSync`, `readFileSync`, etc.).
**Rationale:**
- Unit tests can run without touching the real filesystem, `PATH`, or subprocess layer.
- The CLI itself injects its real dependencies through default parameters, so production behavior is unchanged.
- There is no global mocking required; each test provides its own narrow fakes.
### Minimal Dependency Surface
The runtime dependency graph contains exactly one external package: `minimist` (argument parsing). Everything else uses Node.js built-ins (`child_process`, `fs`, `os`, `path`).
**Rationale:**
- Reduces supply-chain risk and install time.
- Avoids version-lock issues across Node.js 20+ environments.
- Keeps the compiled/bundled footprint negligible for a tool that is often installed as a sidecar.
## Extension Points
### Adding a New Client
To support a fourth (or fifth) AI CLI client, change four files in `src/` and the corresponding tests:
1. **`src/types.ts`** — Add the new name to the `ClientName` union type.
2. **`src/constants.ts`** — Append the new name to `CLIENT_NAMES`.
3. **`src/execute.ts`** — Add an entry to `CLIENT_ARGS` with the clients native argument shape.
4. **`src/config.ts`** — No change required; the existing loop over `CLIENT_NAMES` automatically picks up the new env/flag/file keys.
5. **`src/dispatch.ts`** — Add a keyword check for the new client in `resolveClient`. Decide its precedence relative to existing keywords.
6. **Tests** — Add colocated test cases in `tests/dispatch.test.ts`, `tests/execute.test.ts`, and `tests/detect.test.ts`.
No changes are needed in `cli.ts` because it iterates over `CLIENT_NAMES` for validation.
### Streaming Support
If a future use case requires real-time output (e.g., long-running codegen with progressive feedback), the cleanest extension is to add an optional `onData` callback to `ExecuteOptions`:
```typescript
export interface ExecuteOptions {
clientPath?: string;
timeoutMs?: number;
spawn?: ...;
existsSync?: ...;
onData?: (chunk: string, stream: "stdout" | "stderr") => void;
}
```
When `onData` is provided, `executePrompt` would:
- Continue buffering internally for the final `ExecResult`.
- Also emit each chunk through `onData` so the caller can stream to a UI or logger.
- Reject/resolve with the same error taxonomy.
This preserves backward compatibility: existing callers that omit `onData` receive the exact same buffered `ExecResult` they get today.
### Platform Backends
The current Windows support is limited to discovery (`where` instead of `which`, `.exe` extension assumptions). If future clients require platform-specific spawn options (e.g., PowerShell quoting rules), the extension point is `CLIENT_ARGS` or a new `CLIENT_SPAWN_OPTIONS` record keyed by `ClientName`.
## Testing Strategy
The test suite in `tests/` mirrors the `src/` structure:
| Test file | Coverage |
|---|---|
| `cli.test.ts` | Argument parsing, command routing, JSON/text output modes, exit codes, error formatting |
| `config.test.ts` | Layered precedence of flags, env, file, and `which` fallback; malformed JSON tolerance |
| `detect.test.ts` | `which` success/failure, PATH directory fallback, version parsing, missing binary handling |
| `dispatch.test.ts` | Keyword matching, case insensitivity, `--client` precedence, `defaultClient` fallback, invalid flag handling |
| `execute.test.ts` | Successful execution, stderr capture, non-zero exit codes, `ENOENT``ClientNotFoundError`, timeout, empty prompt rejection, special-character preservation |
All tests use injected mocks; no test spawns real client binaries or reads the real filesystem.
-41
View File
@@ -1,41 +0,0 @@
# elevenlabs-stt
Transcribe local audio files with ElevenLabs Speech-to-Text.
## What this skill is for
- Local audio transcription
- Voice note transcription
- Optional speaker diarization
- Language hints and event tagging
- JSON output for programmatic use
## Requirements
Required binaries:
- `curl`
- `jq`
- `python3`
Preferred auth:
- `ELEVENLABS_API_KEY` in the environment
Fallback auth:
- local OpenClaw config lookup from `~/.openclaw/openclaw.json` or `~/.openclaw/secrets.json`
## Wrapper
Use the bundled script directly:
```bash
bash skills/elevenlabs-stt/scripts/transcribe.sh /path/to/audio.mp3
bash skills/elevenlabs-stt/scripts/transcribe.sh /path/to/audio.mp3 --diarize --lang en
bash skills/elevenlabs-stt/scripts/transcribe.sh /path/to/audio.mp3 --json
bash skills/elevenlabs-stt/scripts/transcribe.sh /path/to/audio.mp3 --events
```
## Notes
- Uses ElevenLabs STT model `scribe_v2`.
- Uploads a local file directly to ElevenLabs.
- If `ELEVENLABS_API_KEY` is not exported, the script tries local OpenClaw config/secrets automatically.
+77
View File
@@ -0,0 +1,77 @@
# flight-finder
Reusable flight-search report skill for OpenClaw. It replaces the brittle one-off `dfw-blq-2026.md` prompt with typed intake, bounded source orchestration, explicit report gates, fixed-template PDF output, and Luke-sender email delivery.
## Core behavior
`flight-finder` is designed to:
- collect missing trip inputs explicitly instead of relying on hardcoded prompt prose
- support split returns, flexible dates, passenger groups, and exclusions
- use bounded search phases across KAYAK, Skyscanner, Expedia, and a best-effort airline direct cross-check
- normalize pricing to USD before ranking
- produce a report payload first, then render PDF/email only when the report is complete
- render booking links in the PDF for each recommended fare, including airline-direct links when they were actually captured
- behave safely on WhatsApp-style chat surfaces by treating status nudges as updates, not resets
- require a fresh bounded search run for every report instead of reusing earlier captures
## Important rules
- Recipient email is a delivery gate, not a search gate.
- Cached flight-search data is forbidden as primary evidence for a new run.
- same-day workspace captures must not be reused
- previously rendered PDFs must not be treated as fresh search output
- if fresh search fails, the skill must degrade honestly instead of recycling earlier artifacts
- `marketCountry` is explicit-only in this implementation pass.
- It must be an ISO 3166-1 alpha-2 uppercase code such as `TH` or `DE`.
- If present, it activates VPN only for the bounded search phase.
- If omitted, no VPN change happens.
- If VPN connect or verification fails, the run falls back to the default market and records a degraded warning instead of hanging.
- Direct-airline cross-checking is currently best-effort, not a hard blocker.
- The PDF should show:
- the captured booking/search link for every quoted fare
- the airline-direct link only when `directBookingUrl` was actually captured
- an explicit “not captured in this run” note when direct-airline booking was unavailable
## DFW ↔ BLQ daily automation
- The 6 AM DFW ↔ BLQ automation should invoke `flight-finder` directly with the structured JSON payload.
- The legacy file at `workspace/docs/prompts/dfw-blq-2026.md` should mirror that exact direct skill invocation, not old prose instructions.
## Helper package
From `skills/flight-finder/`:
```bash
npm install
npm run normalize-request -- --legacy-dfw-blq
npm run normalize-request -- --input "<request.json>"
npm run report-status -- --input "<report-payload.json>"
npm run render-report -- --input "<report-payload.json>" --output "<report.pdf>"
npm run delivery-plan -- --to "<recipient@example.com>" --subject "<subject>" --body "<body>" --attach "<report.pdf>"
```
Expected report-payload provenance fields:
- `searchExecution.freshSearch: true`
- `searchExecution.startedAt`
- `searchExecution.completedAt`
- `searchExecution.artifactsRoot`
## Delivery
- sender identity: `luke@fiorinis.com`
- send path:
- `zsh ~/.openclaw/workspace/bin/gog-luke gmail send --to "<target>" --subject "<subject>" --body "<body>" --attach "<pdf-path>"`
- if the user already provided the destination email, that counts as delivery authorization once the report is ready
## Source viability for implementation pass 1
Bounded checks on Stefano's MacBook Air showed:
- `KAYAK`: viable
- `Skyscanner`: viable
- `Expedia`: viable
- airline direct-booking cross-check: degraded / best-effort
That means the first implementation pass should rely primarily on the three aggregator sources and treat direct-airline confirmation as additive evidence when it succeeds.
+7
View File
@@ -41,7 +41,14 @@ Optional env:
```bash
node integrations/google-workspace/gw.js whoami
node integrations/google-workspace/gw.js send --to "user@example.com" --subject "Hello" --body "Hi there"
node integrations/google-workspace/gw.js send --to "user@example.com" --subject "Report" --body "Attached is the PDF." --attach /tmp/report.pdf
node integrations/google-workspace/gw.js search-mail --query "from:someone@example.com newer_than:7d" --max 10
node integrations/google-workspace/gw.js search-calendar --timeMin 2026-03-17T00:00:00-05:00 --timeMax 2026-03-18T00:00:00-05:00 --max 20
node integrations/google-workspace/gw.js create-event --summary "Meeting" --start 2026-03-20T09:00:00-05:00 --end 2026-03-20T10:00:00-05:00
```
## Attachments
- `send` now supports one or more `--attach /path/to/file` arguments.
- Attachment content type is inferred from the filename extension; PDF attachments are sent as `application/pdf`.
- The default impersonation remains `stefano@fiorinis.com`, so this is the correct helper for actions explicitly performed on Stefano's behalf.
+219
View File
@@ -0,0 +1,219 @@
# Installation
This page covers installing the `ai-cli-dispatch` tool, its prerequisites, and post-install verification.
## Prerequisites
- **Node.js** ≥ 20 (required for `tsx`, `import` attributes, and modern `node:child_process` APIs)
- **npm** (bundled with Node.js)
- **Homebrew** (macOS/Linux) — recommended for installing the underlying AI CLI clients (`codex`, `claude`, `opencode`)
- One or more supported AI CLI clients:
- [Codex CLI](https://github.com/openai/codex)
- [Claude Code](https://docs.anthropic.com/en/docs/agents-and-tools/claude-code/overview)
- [OpenCode](https://github.com/opencode-ai/opencode)
## Install the tool
### 1. Clone the repository
```bash
git clone <repository-url> ~/.openclaw/workspace/skills/ai-cli-dispatch
cd ~/.openclaw/workspace/skills/ai-cli-dispatch
```
If you already have the full `stef-openclaw-skills` repo checked out, use the path inside it instead:
```bash
cd ~/.openclaw/workspace/skills/stef-openclaw-skills/tools/ai-cli-dispatch
```
### 2. Install Node dependencies
```bash
npm install
```
This installs:
- `tsx` — TypeScript execution runtime
- `minimist` — argument parsing
- `typescript` — type checking
## PATH configuration
The helper script lives at `scripts/ai-cli-dispatch`. Add it to your shell PATH so OpenClaw (or your terminal) can invoke it without a full path:
```bash
# ~/.zshrc or ~/.bashrc
export PATH="$HOME/.openclaw/workspace/skills/ai-cli-dispatch/scripts:$PATH"
```
Reload your shell:
```bash
source ~/.zshrc # or ~/.bashrc
```
Verify the script is reachable:
```bash
which ai-cli-dispatch
ai-cli-dispatch --help
```
## Optional configuration file
Create `~/.openclaw/ai-cli-dispatch.json` to customize client paths and set a default client:
```bash
mkdir -p ~/.openclaw
$EDITOR ~/.openclaw/ai-cli-dispatch.json
```
Example configuration:
```json
{
"paths": {
"codex": "/opt/homebrew/bin/codex",
"claude": "/opt/homebrew/bin/claude",
"opencode": "/opt/homebrew/bin/opencode"
},
"defaultClient": "claude"
}
```
### Configuration precedence
When resolving a client binary, the tool checks sources in this order (first match wins):
1. CLI flag: `--codex-path`, `--claude-path`, `--opencode-path`
2. Environment variable: `AI_CLI_CODEX_PATH`, `AI_CLI_CLAUDE_PATH`, `AI_CLI_OPENCODE_PATH`
3. File config: `paths.<client>` in `~/.openclaw/ai-cli-dispatch.json`
4. System `PATH` lookup via `which` / `where`
The `defaultClient` follows the same precedence:
1. CLI flag: `--default-client`
2. Environment variable: `AI_CLI_DEFAULT_CLIENT`
3. File config: `defaultClient` in `~/.openclaw/ai-cli-dispatch.json`
## Install AI CLI clients
### Codex
```bash
npm install -g @openai/codex
```
### Claude Code
```bash
npm install -g @anthropic-ai/claude-code
```
### OpenCode
```bash
npm install -g @opencode-ai/opencode
```
Or via Homebrew where formulas are available:
```bash
brew install codex # if available in your tap
brew install claude-code # if available in your tap
```
## Verification
### 1. Check local tool health
```bash
ai-cli-dispatch --help
```
Expected output:
```text
AI CLI Dispatch
Usage:
ai-cli-dispatch list [--json|--text]
ai-cli-dispatch run --client <client> --prompt <prompt> [--json|--text]
ai-cli-dispatch dispatch <prompt> [--client <client>] [--json|--text]
ai-cli-dispatch --help
Clients: codex, claude, opencode
```
### 2. List discovered clients
```bash
ai-cli-dispatch list --json
```
Example output when two clients are installed:
```json
[
{
"name": "codex",
"path": "/opt/homebrew/bin/codex",
"version": "1.2.3",
"found": true
},
{
"name": "claude",
"path": "/opt/homebrew/bin/claude",
"version": "0.7.8",
"found": true
},
{
"name": "opencode",
"found": false
}
]
```
### 3. Run a quick dispatch
```bash
ai-cli-dispatch run --client codex --prompt "hello" --json
```
This should return a JSON result with `stdout`, `stderr`, and `exitCode`.
### 4. Test keyword dispatch
```bash
ai-cli-dispatch dispatch "refactor this using claude"
```
The tool inspects the prompt for client keywords (`claude`, `codex`, `opencode`, `open code`) and routes to the matching client.
## Troubleshooting
### `Missing local Node dependencies for ai-cli-dispatch`
Run `npm install` from the skill directory:
```bash
cd ~/.openclaw/workspace/skills/ai-cli-dispatch
npm install
```
### `Client "codex" not found or not installed`
- Ensure the client is installed globally or via Homebrew
- Verify it is on your PATH: `which codex`
- Or override the path in `~/.openclaw/ai-cli-dispatch.json` or with an environment variable
### `Prompt cannot be empty`
The `run` and `dispatch` commands require a non-empty `--prompt` or trailing prompt text.
### Config file is not being read
- Verify the file is at exactly `~/.openclaw/ai-cli-dispatch.json`
- Check for JSON syntax errors (trailing commas are not allowed)
- Use `--debug` for deeper troubleshooting if supported by the calling context
+384
View File
@@ -0,0 +1,384 @@
# nordvpn-client
Cross-platform NordVPN lifecycle skill for macOS and Linux.
## Overview
`nordvpn-client` is the operator-facing VPN control skill for OpenClaw. It can:
- detect whether the host is ready for NordVPN automation
- install or bootstrap the required backend
- validate auth
- connect to a target country or city
- verify the public exit location
- disconnect and restore normal local networking state
The skill uses different backends by platform:
- Linux: official `nordvpn` CLI
- macOS: NordLynx/WireGuard with `wireguard-go` and `wireguard-tools`
## Commands
```bash
node skills/nordvpn-client/scripts/nordvpn-client.js status
node skills/nordvpn-client/scripts/nordvpn-client.js install
node skills/nordvpn-client/scripts/nordvpn-client.js login
node skills/nordvpn-client/scripts/nordvpn-client.js verify
node skills/nordvpn-client/scripts/nordvpn-client.js verify --country "Italy"
node skills/nordvpn-client/scripts/nordvpn-client.js verify --country "Italy" --city "Milan"
node skills/nordvpn-client/scripts/nordvpn-client.js connect --country "Italy"
node skills/nordvpn-client/scripts/nordvpn-client.js connect --city "Tokyo"
node skills/nordvpn-client/scripts/nordvpn-client.js connect --country "Japan" --city "Tokyo"
node skills/nordvpn-client/scripts/nordvpn-client.js disconnect
node skills/nordvpn-client/scripts/nordvpn-client.js status --debug
```
## Credentials
Supported inputs:
- `NORDVPN_TOKEN`
- `NORDVPN_TOKEN_FILE`
- `NORDVPN_USERNAME`
- `NORDVPN_PASSWORD`
- `NORDVPN_PASSWORD_FILE`
Default OpenClaw credential paths:
- token: `~/.openclaw/workspace/.clawdbot/credentials/nordvpn/token.txt`
- password: `~/.openclaw/workspace/.clawdbot/credentials/nordvpn/password.txt`
Recommended setup on macOS is a token file with strict permissions:
```bash
mkdir -p ~/.openclaw/workspace/.clawdbot/credentials/nordvpn
chmod 700 ~/.openclaw/workspace/.clawdbot/credentials/nordvpn
printf '%s\n' '<your-nordvpn-token>' > ~/.openclaw/workspace/.clawdbot/credentials/nordvpn/token.txt
chmod 600 ~/.openclaw/workspace/.clawdbot/credentials/nordvpn/token.txt
```
Do not commit secrets into the repo or the skill docs.
## Platform Backends
### macOS
Current macOS backend:
- NordLynx/WireGuard
- `wireguard-go`
- `wireguard-tools`
- explicit macOS DNS management on eligible physical services:
- `103.86.96.100`
- `103.86.99.100`
Important behavior:
- `NordVPN.app` may remain installed, but the automated backend does not reuse app login state.
- the generated WireGuard config intentionally stays free of `DNS = ...` so `wg-quick` does not rewrite every macOS network service behind the skills back.
- during `connect`, the skill first proves the tunnel is stable with a bounded persistence gate that reuses the allowed helper `probe` action and a verified public exit.
- during `connect`, the skill snapshots current DNS/search-domain settings on eligible physical services and then applies NordVPN DNS only after that stable gate, one last liveness check, and a post-DNS system-hostname-resolution check succeed.
- during `disconnect`, or after a failed/stale teardown, the skill restores the saved DNS/search-domain snapshot.
- if persistence, exit verification, or post-DNS hostname resolution fails, the skill rolls back before treating the connect as successful and resumes Tailscale if it stopped it.
- when the skill intentionally stops Tailscale for a VPN session, it writes a short-lived suppression marker so host watchdogs do not immediately run `tailscale up` and fight the VPN route change.
- The skill automatically suspends Tailscale before connect if Tailscale is active.
- The skill resumes Tailscale after disconnect, or after a failed connect, if it stopped it.
- The Homebrew NordVPN app does not need to be uninstalled.
### Linux
Current Linux backend:
- official `nordvpn` CLI
- official NordVPN installer
- token login through `nordvpn login --token ...`
## Install / Bootstrap
### macOS
Bootstrap the automation backend:
```bash
node skills/nordvpn-client/scripts/nordvpn-client.js install
```
Equivalent Homebrew command:
```bash
brew install wireguard-go wireguard-tools
```
What `install` does on macOS:
- checks whether `wireguard-go` is present
- checks whether `wg` and `wg-quick` are present
- installs missing packages through Homebrew
### Linux
```bash
node skills/nordvpn-client/scripts/nordvpn-client.js install
```
What `install` does on Linux:
- downloads NordVPNs official installer script
- runs it
- leaves subsequent login/connect to the official `nordvpn` CLI
## macOS sudoers Setup
Automated macOS connect/disconnect requires passwordless `sudo` for the helper script that invokes `wg-quick`.
Installed OpenClaw helper path:
```text
/Users/stefano/.openclaw/workspace/skills/nordvpn-client/scripts/nordvpn-wireguard-helper.sh
```
Edit sudoers safely:
```bash
sudo visudo
```
Add this exact rule:
```sudoers
stefano ALL=(root) NOPASSWD: /Users/stefano/.openclaw/workspace/skills/nordvpn-client/scripts/nordvpn-wireguard-helper.sh probe, /Users/stefano/.openclaw/workspace/skills/nordvpn-client/scripts/nordvpn-wireguard-helper.sh up, /Users/stefano/.openclaw/workspace/skills/nordvpn-client/scripts/nordvpn-wireguard-helper.sh down
```
Do not add extra helper actions just for persistence checks unless you are also updating host sudoers. The current implementation intentionally rides the persistence check on `probe` so the existing `probe/up/down` rule remains sufficient.
If you run the repo copy directly instead of the installed OpenClaw skill, adjust the helper path accordingly.
## Common Flows
### Status
```bash
node skills/nordvpn-client/scripts/nordvpn-client.js status
```
Use this first to answer:
- is the correct backend available?
- is the token visible?
- is `sudoReady` true?
- is the machine currently connected?
### Login
```bash
node skills/nordvpn-client/scripts/nordvpn-client.js login
```
On macOS this validates the token and populates the local auth cache. It does not connect the VPN.
### Connect
Country:
```bash
node skills/nordvpn-client/scripts/nordvpn-client.js connect --country "Germany"
```
City:
```bash
node skills/nordvpn-client/scripts/nordvpn-client.js connect --country "Japan" --city "Tokyo"
```
Expected macOS behavior:
- stop Tailscale if active
- select a NordVPN server for the target
- bring up the WireGuard tunnel
- prove persistence of the live `utun*` runtime via the helper `probe` path
- verify the public exit location
- run one final liveness check before applying NordVPN DNS
- return JSON describing the chosen server and final verified location
### Verify
```bash
node skills/nordvpn-client/scripts/nordvpn-client.js verify --country "Germany"
```
Use this after connect if you want an explicit location check without changing VPN state.
### Disconnect
```bash
node skills/nordvpn-client/scripts/nordvpn-client.js disconnect
```
Expected macOS behavior:
- attempt `wg-quick down` whenever there is active or residual NordVPN WireGuard state
- remove stale local NordVPN state files after teardown
- restore automatic DNS when the saved DNS snapshot is obviously just NordVPN-pinned leftovers
- resume Tailscale if the skill had suspended it
## Output Model
Normal JSON is redacted by default.
Redacted fields in normal mode:
- `cliPath`
- `appPath`
- `wireguard.configPath`
- `wireguard.helperPath`
- `wireguard.authCache.tokenSource`
Operational fields preserved in normal mode:
- `connected`
- `wireguard.active`
- `wireguard.endpoint`
- `requestedTarget`
- `verification`
- public IP and location
For deeper troubleshooting, use:
```bash
node skills/nordvpn-client/scripts/nordvpn-client.js status --debug
```
`--debug` keeps the internal local paths, helper-hardening diagnostics, and other low-level metadata in the JSON output.
If you also run local watchdogs such as `healthwatch.sh`, they should honor the NordVPN Tailscale suppression marker at `~/.nordvpn-client/tailscale-suppressed` and skip automatic `tailscale up` while the marker is fresh or the NordVPN WireGuard tunnel is active.
## Troubleshooting
### `Invalid authorization header`
Meaning:
- the token file was found
- the token value is not valid for NordVPNs API
Actions:
1. generate a fresh NordVPN access token
2. replace the contents of `~/.openclaw/workspace/.clawdbot/credentials/nordvpn/token.txt`
3. run:
```bash
node skills/nordvpn-client/scripts/nordvpn-client.js login
```
### `sudoReady: false`
Meaning:
- the helper script is present
- the agent cannot run `wg-quick` non-interactively
Actions:
1. add the `visudo` rule shown above
2. rerun:
```bash
node skills/nordvpn-client/scripts/nordvpn-client.js status
```
Expected:
- `wireguard.sudoReady: true`
### WireGuard tools missing
Meaning:
- macOS backend is selected
- `wireguard-go`, `wg`, or `wg-quick` is missing
Actions:
```bash
node skills/nordvpn-client/scripts/nordvpn-client.js install
```
or:
```bash
brew install wireguard-go wireguard-tools
```
### Tailscale interaction
Expected behavior on macOS:
- Tailscale is suspended before the NordVPN connect
- Tailscale is resumed after disconnect or failed connect
If a connect succeeds but later traffic is wrong, check:
```bash
node skills/nordvpn-client/scripts/nordvpn-client.js status
/opt/homebrew/bin/tailscale status --json
```
Look for:
- `connected: true` and a foreign exit IP while NordVPN is up
- `connected: false` and Texas/Garland IP after disconnect
### Status says disconnected after a verified connect
This was a previous macOS false-negative path and is now normalized in the connect response.
Current expectation:
- if `connect` verifies the target location successfully
- the returned `state` snapshot should also show:
- `connected: true`
- `wireguard.active: true`
If that regresses, capture:
- `connect` JSON
- `verify` JSON
- `status --debug` JSON
### Disconnect says “no active connection” but traffic is still foreign
The current macOS disconnect path now treats residual WireGuard state as sufficient reason to attempt teardown.
Safe operator check:
```bash
node skills/nordvpn-client/scripts/nordvpn-client.js disconnect
node skills/nordvpn-client/scripts/nordvpn-client.js verify
```
Expected after a good disconnect:
- Texas/Garland public IP again
- `wireguard.configPath: null` in normal status output
- `wireguard.lastConnection: null`
If that regresses, capture:
- `disconnect` JSON
- `verify` JSON
- `status --debug` JSON
## Recommended Agent Workflow
For VPN-routed work:
1. `status`
2. `install` if backend tooling is missing
3. `login` if token validation has not happened yet
4. `connect --country ...` or `connect --country ... --city ...`
5. `verify`
6. run the follow-up skill such as `web-automation`
7. `disconnect`
8. `verify` again if you need proof the machine returned to the normal exit path
+375
View File
@@ -0,0 +1,375 @@
# OpenClaw ACP Orchestration
This document describes the local OpenClaw ACP setup used to orchestrate Codex and Claude Code from an OpenClaw agent on the gateway machine.
## Scope
The target workflow is:
- OpenClaw remains the orchestration brain
- natural-language requests like `use codex for this` or `run this in claude code` are routed to ACP
- the coding harness runs on the same gateway machine where the local `codex` and `claude` clients are installed
- session lifecycle is handled through OpenClaw ACP rather than sub-agents or shell relay hacks
## Local Baseline Before ACP Enablement
Captured on 2026-03-29:
- OpenClaw: `2026.3.28 (f9b1079)`
- bundled `acpx` plugin present locally but disabled and not in the plugin allowlist
- local `codex`: `/opt/homebrew/bin/codex` `0.117.0`
- local `claude`: `/opt/homebrew/bin/claude` `2.1.87`
- gateway host: 8 CPU cores, 8 GB RAM
- default OpenClaw agent workspace: `~/.openclaw/workspace`
## Architectural Decision
Primary architecture:
- OpenClaw ACP with `acpx`
Fallback architecture only if parity is not acceptable:
- `openclaw mcp serve` with Codex or Claude Code connected as external MCP clients to existing OpenClaw channel conversations
Why ACP is primary:
- this is the official OpenClaw architecture for "run this in Codex" / "start Claude Code in a thread"
- it gives durable ACP sessions, resume, bindings, and programmatic `sessions_spawn runtime:"acp"`
## Important Runtime Caveat
The bundled `acpx` runtime supports Codex and Claude, but the stock aliases are adapter commands, not necessarily the bare local terminal binaries:
- `codex -> npx -y @zed-industries/codex-acp@0.9.5`
- `claude -> npx -y @zed-industries/claude-agent-acp@0.21.0`
That means "same as terminal" behavior has to be validated explicitly. It is not guaranteed just because ACP works.
## Baseline Configuration Applied
The current host-local OpenClaw config keeps the native `main` orchestrator and adds ACP-backed agents alongside it:
- `agents.list[0] = main` with `runtime.type = "embedded"`
- `agents.list[1] = codex` with `runtime.type = "acp"`
- `agents.list[2] = claude` with `runtime.type = "acp"`
- `acp.enabled = true`
- `acp.dispatch.enabled = true`
- `acp.backend = "acpx"`
- `acp.defaultAgent = "codex"`
- `acp.allowedAgents = ["claude", "codex"]`
- `acp.maxConcurrentSessions = 2`
- `plugins.allow += acpx`
- `plugins.entries.acpx.enabled = true`
- ACP-specific `cwd` values are absolute paths, not `~`-prefixed shortcuts
The `main` entry is intentional. Once `agents.list` is populated, OpenClaw treats that list as the agent inventory. If `main` is omitted, ACP targets can displace the native orchestrator and break the intended architecture.
## ACP Health Equivalents
The docs mention `/acp doctor`, but the operator-friendly local equivalents on this host are:
- `openclaw config validate`
- `openclaw plugins inspect acpx --json`
- `openclaw gateway status --json`
- `openclaw status --deep`
- `cd /opt/homebrew/lib/node_modules/openclaw/dist/extensions/acpx && ./node_modules/.bin/acpx config show`
Healthy baseline on this machine means:
- config validates
- `acpx` plugin status is `loaded`
- gateway RPC is healthy
- `openclaw status --deep` shows `Agents 3` with `default main`
- `acpx config show` works without bootstrap errors
- `plugins.installs` does not need an `acpx` record because `acpx` is bundled with OpenClaw, not separately installed
Important health nuance:
- `openclaw plugins inspect acpx --json` only tells you the plugin is loaded, not that the ACP backend is healthy enough for `sessions_spawn runtime:"acp"`
- the actual readiness signal is the gateway log line `acpx runtime backend ready`
- during rollout, the backend stayed unavailable until ACP-specific `cwd` values were changed from `~/.openclaw/workspace` to absolute paths
- a later startup bug showed that pinning a custom `command` path disables the plugin-local managed install path and can leave ACP unavailable after a restart if the local `acpx` artifact is absent at boot
- the current host fix is to leave `plugins.entries.acpx.config.command` unset so the bundled plugin can manage its own plugin-local `acpx` binary
Maintenance note:
- the current host intentionally uses the managed plugin-local default command path rather than a custom override
- after any OpenClaw upgrade, re-run:
- `openclaw config validate`
- `openclaw plugins inspect acpx --json`
- `openclaw logs --limit 80 --plain --timeout 10000 | rg 'acpx runtime backend (registered|ready|probe failed)'`
- `ls -l /opt/homebrew/lib/node_modules/openclaw/dist/extensions/acpx/node_modules/.bin/acpx`
- if ACP comes up unavailable at startup, check whether a custom `plugins.entries.acpx.config.command` override was reintroduced before debugging deeper
## Security Review
### Why this needs a review
ACP coding sessions are headless and non-interactive. If they are allowed to write files and run shell commands, the permission mode matters a lot.
### Leading rollout candidate
- `plugins.entries.acpx.config.permissionMode = "approve-all"`
- `plugins.entries.acpx.config.nonInteractivePermissions = "deny"`
Why `deny` instead of `fail`:
- on this host, graceful degradation is better than crashing an otherwise useful ACP session at the first blocked headless permission prompt
- the live `acpx` plugin schema for OpenClaw `2026.3.28` validates `deny`, so this is an intentional runtime choice rather than a placeholder
### What `approve-all` means here
On this gateway host, an ACP coding harness may:
- write files in the configured working tree
- execute shell commands without an interactive prompt
- access network resources that are already reachable from the host
- read local home-directory configuration that the launched harness itself can reach
### Risk boundaries
This host already runs OpenClaw with:
- `tools.exec.host = "gateway"`
- `tools.exec.security = "full"`
- `tools.exec.ask = "off"`
So ACP `approve-all` does not create the first fully trusted execution path on this machine. It extends that trust to ACP-backed Codex/Claude sessions. That is still a meaningful trust expansion and should stay limited to trusted operators and trusted channels.
### First-wave rollout stance
Recommended first wave:
- enable ACP only for trusted direct operators
- prefer explicit `agentId` routing and minimal bindings
- defer broad persistent group bindings until parity and lifecycle behavior are proven
- keep the plugin-tools bridge off unless there is a proven need for ACP harnesses to call OpenClaw plugin tools from inside the session
## Observability And Recovery
Minimum required operational checks:
- `openclaw config validate`
- `openclaw plugins inspect acpx --json`
- `openclaw gateway status --json`
- `openclaw status --deep`
- `openclaw logs --follow`
- `/tmp/openclaw/openclaw-YYYY-MM-DD.log`
Operational questions this setup must answer:
- did an ACP session start
- which harness was used
- which session key is active
- where a stall or permission denial first occurred
- whether the gateway restart preserved resumable state
Current host signals:
- plugin status: `openclaw plugins inspect acpx --json`
- gateway/runtime health: `openclaw gateway status --json`
- agent inventory and active session count: `openclaw status --deep`
- ACP adapter defaults and override file discovery: `acpx config show`
- first runtime failure point: gateway log under `/tmp/openclaw/`
Claude adapter noise:
- the Claude ACP adapter currently emits `session/update` validation noise for `usage_update` after otherwise successful turns
- when filtering logs during Claude ACP troubleshooting, separate that known noise from startup failures by focusing first on:
- `acpx runtime backend ready`
- `ACP runtime backend is currently unavailable`
- `probe failed`
- actual session spawn/close lines
## Concurrency Stance
This machine has 8 CPU cores and 8 GB RAM. A conservative initial ACP concurrency cap is better than the plan's generic placeholder of `8`.
Recommended initial cap:
- `acp.maxConcurrentSessions = 2`
Reason:
- enough for one Codex and one Claude session at the same time
- low enough to reduce memory pressure and noisy contention on the same laptop-class host
- if operators start using longer-lived persistent ACP sessions heavily, revisit this only after checking real memory pressure and swap behavior on the gateway host
## Plugin Tools Bridge
The planning material discussed `plugins.entries.acpx.config.pluginToolsMcpBridge`, but the local `2026.3.28` bundled `acpx` schema does not currently expose that key in `openclaw plugins inspect acpx --json`.
Current stance:
- treat plugin-tools bridge as unsupported unless the live runtime proves otherwise
- do not add that key blindly to `openclaw.json`
## Default Workspace Root
The default ACP workspace root for this install is:
- `~/.openclaw/workspace`
Per-session or per-binding `cwd` values can narrow from there when a specific repository or skill workspace is known.
For ACP plugin/runtime config, use absolute paths instead of `~`-prefixed paths.
## Parity Results
### Codex ACP parity
Validated directly with `acpx codex` against a real project worktree.
Observed:
- correct `cwd`
- `HOME=/Users/stefano`
- access to `~/.codex`
- access to `~/.openclaw/workspace`
- access to installed Codex skills under `~/.codex/skills`
- persistent named sessions retained state across turns
- persistent named sessions retained state across an OpenClaw gateway restart
Assessment:
- Codex ACP is close enough to local terminal behavior for rollout
### Claude Code ACP parity
Validated directly with `acpx claude` against the same project worktree.
Observed:
- correct `cwd`
- `HOME=/Users/stefano`
- access to `~/.claude`
- access to `~/.codex` when explicitly tested with shell commands
- persistent named sessions retained state across turns
- persistent named sessions retained state across an OpenClaw gateway restart
Known defect:
- the Claude ACP adapter emits an extra `session/update` validation error after otherwise successful turns:
- `Invalid params`
- `sessionUpdate: 'usage_update'`
Assessment:
- Claude ACP is usable, but noisier than Codex
- this is an adapter/protocol mismatch to monitor, not a rollout blocker for trusted operators
## ACPX Override Decision
Decision:
- do **not** add `~/.acpx/config.json` agent overrides for Codex or Claude right now
Why:
- Codex parity already passes with the stock alias path
- swapping Claude from the deprecated package name to `@agentclientprotocol/claude-agent-acp@0.24.2` did **not** remove the `session/update` validation noise
- raw local `codex` and `claude` CLIs are not drop-in ACP servers, so an override would add maintenance cost without delivering materially better parity
## Natural-Language Routing Policy
The `main` agent is instructed to:
- stay native as the orchestrator
- use `sessions_spawn` with `runtime: "acp"` when the user explicitly asks for Codex or Claude Code
- choose `agentId: "codex"` or `agentId: "claude"` accordingly
- use one-shot ACP runs for single tasks
- use persistent ACP sessions only when the user clearly wants continued context
- avoid silent fallback to ordinary local exec when ACP was explicitly requested
The live messaging tool surface had to be extended to expose:
- `sessions_spawn`
- `sessions_yield`
without widening the whole profile beyond what was needed.
## Binding Policy
First-wave binding policy is intentionally conservative:
- no broad top-level persistent `bindings[]`
- no automatic permanent channel/topic binds
- prefer on-demand ACP spawn from the current conversation
- only introduce persistent binds later if there is a clear operator need
Channel-specific note:
- WhatsApp does not support ACP thread-bound spawn in the tested path
- use current-conversation or one-shot ACP behavior there, not thread-bound ACP assumptions
## Smoke-Test Findings
What worked:
- direct `acpx codex` runs
- direct `acpx claude` runs
- mixed Codex + Claude ACPX runs in parallel
- persistent ACPX named sessions
- named-session recall after a gateway restart
What failed and why:
- channel-less CLI-driven `openclaw agent` tests can fail ACP spawn with:
- `Channel is required when multiple channels are configured: telegram, whatsapp, bluebubbles`
- this is a context issue, not a backend-registration issue
- synthetic CLI sessions are not a perfect substitute for a real inbound channel conversation when testing current-conversation ACP spawn
Operational interpretation:
- ACP backend + harness parity are good enough for rollout
- final operator confidence should still come from a real inbound Telegram or WhatsApp conversation, not only a synthetic CLI turn
## Fallback Decision
Decision:
- keep ACP via `acpx` as the primary architecture
- do **not** adopt `openclaw mcp serve` as the primary mode at this stage
Why fallback was not adopted:
- Codex parity is good
- Claude parity is acceptable with one known noisy adapter defect
- OpenClaw can now expose the ACP spawn tool in the messaging profile
- the remaining limitation is real channel context for current-conversation spawn, not a fundamental mismatch between ACP and the installed gateway clients
## Rollback
Back up `~/.openclaw/openclaw.json` before any ACP change.
Current ACP implementation backup:
- `~/.openclaw/openclaw.json.bak.pre-acp-implementation-20260329-231818`
Rollback approach:
1. restore the backup config
2. validate config
3. restart the gateway
4. confirm ACP plugin status and channel health
Example rollback:
```bash
cp ~/.openclaw/openclaw.json.bak.pre-acp-implementation-20260329-231818 ~/.openclaw/openclaw.json
openclaw config validate
openclaw gateway restart
openclaw status --deep
```
## Implementation Hazards
Two local quirks were discovered during rollout:
- `openclaw config set` is not safe for parallel writes to the same config file. Concurrent `config set` calls can clobber each other.
- host-local legacy keys can reappear if a write path round-trips older config state. For this rollout, atomic file edits plus explicit validation were safer than chaining many `config set` commands.
## Implementation Notes
This document is updated milestone by milestone as the ACP rollout is implemented and verified.
@@ -0,0 +1,21 @@
# Web Automation Consolidation Design
## Goal
Consolidate `playwright-safe` into `web-automation` so the repo exposes a single web skill. Keep the proven one-shot extractor behavior, rename it to `extract.js`, and remove the separate `playwright-safe` skill and docs.
## Architecture
`web-automation` remains the only published skill. It will expose two capability bands under one skill: one-shot extraction via `scripts/extract.js`, and broader stateful automation via the existing `auth.ts`, `browse.ts`, `flow.ts`, and `scrape.ts` commands. The one-shot extractor will keep the current safe Playwright behavior: single URL, JSON output, bounded stealth/anti-bot handling, and no sandbox-disabling Chromium flags.
## Migration
- Copy the working extractor into `skills/web-automation/scripts/extract.js`
- Update `skills/web-automation/SKILL.md` and `docs/web-automation.md` to describe both one-shot extraction and full automation
- Remove `skills/playwright-safe/`
- Remove `docs/playwright-safe.md`
- Remove README/doc index references to `playwright-safe`
## Verification
- `node skills/web-automation/scripts/extract.js` -> JSON error for missing URL
- `node skills/web-automation/scripts/extract.js ftp://example.com` -> JSON error for invalid scheme
- `node skills/web-automation/scripts/extract.js https://example.com` -> valid JSON result with title/status
- Repo text scan confirms no remaining published references directing users to `playwright-safe`
- Commit, push, and clean up the worktree
@@ -0,0 +1,132 @@
# Web Automation Consolidation Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Consolidate the separate `playwright-safe` skill into `web-automation` and publish a single web skill with both one-shot extraction and broader automation.
**Architecture:** Move the proven safe one-shot extractor into `skills/web-automation/scripts/extract.js`, update `web-automation` docs to expose it as the simple path, and remove the separate `playwright-safe` skill and docs. Keep the extractor behavior unchanged except for its new location/name.
**Tech Stack:** Node.js, Playwright, Camoufox skill docs, git
---
### Task 1: Create isolated worktree
**Files:**
- Modify: repo git metadata only
**Step 1: Create worktree**
Run:
```bash
git -C /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills worktree add /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills/.worktrees/web-automation-consolidation -b feature/web-automation-consolidation
```
**Step 2: Verify baseline**
Run:
```bash
git -C /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills/.worktrees/web-automation-consolidation status --short --branch
```
Expected: clean feature branch
### Task 2: Move the extractor into web-automation
**Files:**
- Create: `skills/web-automation/scripts/extract.js`
- Read: `skills/playwright-safe/scripts/playwright-safe.js`
**Step 1: Copy the extractor**
- Copy the proven script content into `skills/web-automation/scripts/extract.js`
- Adjust only relative paths/messages if needed
**Step 2: Preserve behavior**
- Keep JSON-only output
- Keep URL validation
- Keep stealth/anti-bot behavior
- Keep sandbox enabled
### Task 3: Update skill and docs
**Files:**
- Modify: `skills/web-automation/SKILL.md`
- Modify: `docs/web-automation.md`
- Modify: `README.md`
- Modify: `docs/README.md`
- Delete: `skills/playwright-safe/SKILL.md`
- Delete: `skills/playwright-safe/package.json`
- Delete: `skills/playwright-safe/package-lock.json`
- Delete: `skills/playwright-safe/.gitignore`
- Delete: `skills/playwright-safe/scripts/playwright-safe.js`
- Delete: `docs/playwright-safe.md`
**Step 1: Update docs**
- Make `web-automation` the only published web skill
- Document `extract.js` as the one-shot extraction path
- Remove published references to `playwright-safe`
**Step 2: Remove redundant skill**
- Delete the separate `playwright-safe` skill files and doc
### Task 4: Verify behavior
**Files:**
- Test: `skills/web-automation/scripts/extract.js`
**Step 1: Missing URL check**
Run:
```bash
cd /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills/.worktrees/web-automation-consolidation && node skills/web-automation/scripts/extract.js
```
Expected: JSON error about missing URL
**Step 2: Invalid scheme check**
Run:
```bash
cd /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills/.worktrees/web-automation-consolidation && node skills/web-automation/scripts/extract.js ftp://example.com
```
Expected: JSON error about only http/https URLs allowed
**Step 3: Smoke test**
Run:
```bash
cd /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills/.worktrees/web-automation-consolidation && node skills/web-automation/scripts/extract.js https://example.com
```
Expected: JSON with title `Example Domain`, status `200`, and no sandbox-disabling flags in code
**Step 4: Reference scan**
Run:
```bash
cd /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills/.worktrees/web-automation-consolidation && rg -n "playwright-safe" README.md docs skills
```
Expected: no remaining published references, or only intentional historical plan docs
### Task 5: Commit, push, and clean up
**Files:**
- Modify: git history only
**Step 1: Commit**
Run:
```bash
git add skills/web-automation docs README.md
git commit -m "refactor: consolidate web scraping into web-automation"
```
**Step 2: Push**
Run:
```bash
git push -u origin feature/web-automation-consolidation
```
**Step 3: Merge and cleanup**
- Fast-forward or merge to `main`
- Push `main`
- Remove the worktree
- Delete the feature branch
@@ -0,0 +1,40 @@
# NordVPN Client Skill Design
## Goal
Create a `nordvpn-client` skill that works on macOS and Linux gateway hosts. It should detect whether NordVPN is already installed, bootstrap it if missing, handle login/auth setup, connect to a requested country or city, verify the VPN state and public IP location, disconnect when requested, and then be usable alongside other skills like `web-automation`.
## Architecture
The skill exposes one logical interface with platform-specific backends. Linux uses the official NordVPN CLI path. macOS probes for a usable CLI first, but falls back to the official app workflow when needed. The skill is responsible only for VPN lifecycle and verification, not for wrapping arbitrary commands inside a VPN session.
## Interface
Single script entrypoint:
- `node scripts/nordvpn-client.js install`
- `node scripts/nordvpn-client.js login`
- `node scripts/nordvpn-client.js connect --country "Italy"`
- `node scripts/nordvpn-client.js connect --city "Milan"`
- `node scripts/nordvpn-client.js disconnect`
- `node scripts/nordvpn-client.js status`
## Platform Model
### Linux
- Probe for `nordvpn`
- If missing, bootstrap official NordVPN package/CLI
- Prefer token-based login for non-interactive auth
- Connect/disconnect/status through official CLI
### macOS
- Probe for `nordvpn` CLI if available
- Otherwise probe/install the official app
- Use CLI when present, otherwise automate the app/login flow
- Verify connection using app/CLI state plus external IP/geolocation
## Auth and Safety
- Do not store raw NordVPN secrets in skill docs
- Read token/credentials from env vars or a local credential file path
- Keep the skill focused on install/login/connect/disconnect/status
- After `connect`, verify both local VPN state and external IP/location before the agent proceeds to tasks like `web-automation`
## Verification
- `status` reports platform, install state, auth state, connection state, and public IP/location check
- `connect` verifies the requested target as closely as available data allows
- Local validation happens first in the OpenClaw workspace, then the proven skill is copied into `stef-openclaw-skills`, documented, committed, and pushed
+127
View File
@@ -0,0 +1,127 @@
# NordVPN Client Skill Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Build a cross-platform `nordvpn-client` skill for macOS and Linux that can install/bootstrap NordVPN, log in, connect to a target country or city, verify the VPN session, disconnect, and report status.
**Architecture:** Implement one skill with one script entrypoint and platform-specific backends. Linux uses the official NordVPN CLI. macOS uses a CLI path when present and otherwise falls back to the NordVPN app workflow. The skill manages VPN state only, leaving follow-up operations like `web-automation` to separate agent steps.
**Tech Stack:** Node.js, shell/OS commands, NordVPN CLI/app integration, OpenClaw skills, git
---
### Task 1: Create isolated worktree
**Files:**
- Modify: repo git metadata only
**Step 1: Create worktree**
Run:
```bash
git -C /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills worktree add /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills/.worktrees/nordvpn-client -b feature/nordvpn-client
```
**Step 2: Verify baseline**
Run:
```bash
git -C /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills/.worktrees/nordvpn-client status --short --branch
```
Expected: clean feature branch
### Task 2: Create the local skill runtime
**Files:**
- Create: `skills/nordvpn-client/SKILL.md`
- Create: `skills/nordvpn-client/scripts/nordvpn-client.js`
- Optional Create: helper files under `skills/nordvpn-client/scripts/`
**Step 1: Write the failing checks**
- Missing command/action should fail with clear usage output
- Unsupported platform should fail clearly
**Step 2: Implement platform detection and install probe**
- detect `darwin` vs `linux`
- detect whether NordVPN CLI/app is already present
- expose `status` with install/auth/connect fields
### Task 3: Implement install and auth bootstrap
**Files:**
- Modify: `skills/nordvpn-client/scripts/nordvpn-client.js`
**Step 1: Linux install/login path**
- implement official CLI probe/install path
- implement token-based login path
**Step 2: macOS install/login path**
- probe CLI first
- if absent, probe/install NordVPN app path
- implement login/bootstrap state verification for the app workflow
**Step 3: Keep secrets external**
- env vars or local credential path only
- no raw secrets in docs or skill text
### Task 4: Implement connect/disconnect/status/verification
**Files:**
- Modify: `skills/nordvpn-client/scripts/nordvpn-client.js`
**Step 1: Connect**
- support `--country` and `--city`
- normalize target handling per platform
**Step 2: Verify**
- report local connection state
- run public IP / geolocation verification
- fail if connection target cannot be reasonably verified
**Step 3: Disconnect and status**
- implement clean disconnect
- ensure `status` emits machine-readable output for agent use
### Task 5: Validate locally in OpenClaw workspace
**Files:**
- Test: local workspace copy of `nordvpn-client`
**Step 1: Direct command validation**
- usage errors are correct
- install probe works on this host
- status output is coherent before login/connect
**Step 2: One real connect flow**
- connect to a test country/city if credentials are available
- verify local state + external IP/location
- disconnect cleanly
### Task 6: Promote to repo docs and publish
**Files:**
- Modify: `README.md`
- Modify: `docs/README.md`
- Create: `docs/nordvpn-client.md`
- Create/Modify: `skills/nordvpn-client/...`
**Step 1: Document the skill**
- install/bootstrap behavior
- auth expectations
- connect/disconnect/status commands
- macOS vs Linux notes
**Step 2: Commit and push**
Run:
```bash
git add skills/nordvpn-client docs README.md
git commit -m "feat: add nordvpn client skill"
git push -u origin feature/nordvpn-client
```
**Step 3: Merge and cleanup**
- fast-forward or merge to `main`
- push `main`
- remove the worktree
- delete the feature branch
@@ -0,0 +1,34 @@
# NordVPN macOS WireGuard Backend Design
## Goal
Replace the current macOS app-manual fallback in `nordvpn-client` with a scripted WireGuard/NordLynx backend inspired by `wg-nord` and `wgnord`, while preserving the official Linux `nordvpn` CLI backend.
## Key decisions
- Keep Linux on the official `nordvpn` CLI.
- Prefer a native macOS WireGuard backend over the GUI app.
- Do not vendor third-party scripts directly; reimplement the needed logic in our own JSON-based Node skill.
- Do not require uninstalling the Homebrew `nordvpn` app. The new backend can coexist with it.
## macOS backend model
- Bootstrap via Homebrew:
- `wireguard-tools`
- `wireguard-go`
- Read NordVPN token from existing env/file inputs.
- Discover a WireGuard-capable NordVPN server via the public Nord API.
- Generate a private key locally.
- Exchange the private key for Nord-provided interface credentials using the token.
- Materialize a temporary WireGuard config under a skill-owned state directory.
- Connect and disconnect via `wg-quick`.
- Verify with public IP/geolocation after connect.
## Data/state
- Keep state under a skill-owned directory in the user's home, not `/etc`.
- Persist only what is needed for reconnect/disconnect/status.
- Never store secrets in docs.
## Rollout
1. Implement the macOS WireGuard backend in the skill.
2. Update status output so backend selection is explicit.
3. Update skill docs and repo docs.
4. Verify non-destructive flows on this host.
5. Commit, push, and then decide whether to run a live connect test.
@@ -0,0 +1,11 @@
# NordVPN macOS WireGuard Backend Plan
1. Add a backend selector to `nordvpn-client`.
2. Keep Linux CLI behavior unchanged.
3. Add macOS WireGuard dependency probing and install guidance.
4. Implement token-based NordLynx config generation inspired by `wg-nord`/`wgnord`.
5. Replace the current preferred macOS control mode from `app-manual` to WireGuard when dependencies and token are available.
6. Keep app-manual as the last fallback only.
7. Update `status`, `login`, `connect`, `disconnect`, and `verify` JSON to expose the backend in use.
8. Update repo docs and skill docs to reflect the new model and required token/dependencies.
9. Verify command behavior locally without forcing a live VPN connection unless requested.
@@ -0,0 +1,33 @@
# Web Automation CloakBrowser Migration Design
## Goal
Replace all Camoufox and direct Playwright Chromium usage in `web-automation` with CloakBrowser, while preserving the existing feature set: one-shot extraction, persistent browsing sessions, authenticated flows, multi-step automation, and markdown scraping. After local validation, keep the repo copy and docs as the canonical published version, then commit and push.
## Architecture
`web-automation` will become CloakBrowser-only. A single browser-launch layer in `skills/web-automation/scripts/browse.ts` will provide the canonical runtime for the other scripts. Stateful flows will use CloakBrowser persistent contexts; one-shot extraction will also use CloakBrowser instead of raw `playwright.chromium`.
## Scope
- Replace `camoufox-js` in `browse.ts` and any helper/test scripts
- Replace direct `playwright.chromium` launch in `extract.js`
- Update shared types/imports to match the CloakBrowser Playwright-compatible API
- Remove Camoufox/Chromium-specific setup instructions from skill docs
- Update package metadata and lockfile to depend on `cloakbrowser`
- Keep the user-facing command surface stable where possible
## Compatibility Strategy
To minimize user breakage:
- keep the script filenames and CLI interfaces stable
- support old `CAMOUFOX_*` env vars as temporary aliases where practical
- introduce neutral naming in docs and code for the new canonical path
## Testing Strategy
- Verify launcher setup directly through `browse.ts`
- Verify `extract.js` still handles: missing URL, invalid scheme, smoke extraction from `https://example.com`
- Verify one persistent-context path and one higher-level consumer (`scrape.ts` or `flow.ts`) still works
- Update docs only after the runtime is validated
## Rollout
1. Implement and verify locally in the repo worktree
2. Update repo docs/indexes to describe CloakBrowser-based `web-automation`
3. Commit and push the repo changes
4. If needed, sync the installed OpenClaw workspace copy from the validated repo version
@@ -0,0 +1,136 @@
# Web Automation CloakBrowser Migration Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Replace Camoufox and direct Chromium launches in `web-automation` with CloakBrowser and publish the updated repo/docs.
**Architecture:** Use a single CloakBrowser-backed launch path in `skills/web-automation/scripts/browse.ts`, migrate `extract.js` to the same backend, update dependent scripts and tests, then update docs and publish the repo changes.
**Tech Stack:** Node.js, TypeScript, JavaScript, Playwright-compatible browser automation, CloakBrowser, git
---
### Task 1: Create isolated worktree
**Files:**
- Modify: repo git metadata only
**Step 1: Create worktree**
Run:
```bash
git -C /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills worktree add /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills/.worktrees/web-automation-cloakbrowser -b feature/web-automation-cloakbrowser
```
**Step 2: Verify baseline**
Run:
```bash
git -C /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills/.worktrees/web-automation-cloakbrowser status --short --branch
```
Expected: clean feature branch
### Task 2: Migrate the browser launcher
**Files:**
- Modify: `skills/web-automation/scripts/browse.ts`
- Modify: `skills/web-automation/scripts/package.json`
- Modify: `skills/web-automation/scripts/pnpm-lock.yaml`
- Optional Modify: test helper scripts under `skills/web-automation/scripts/`
**Step 1: Write the failing verification**
Run:
```bash
cd /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills/.worktrees/web-automation-cloakbrowser/skills/web-automation/scripts && node -e "require.resolve('cloakbrowser/package.json')"
```
Expected: fail before dependency migration
**Step 2: Replace backend dependency**
- remove `camoufox-js`
- add `cloakbrowser`
- update `browse.ts` to launch CloakBrowser contexts
- preserve persistent profile support
**Step 3: Verify launcher wiring**
Run a direct browse smoke test after install/update.
### Task 3: Migrate dependent scripts
**Files:**
- Modify: `skills/web-automation/scripts/extract.js`
- Modify: `skills/web-automation/scripts/auth.ts`
- Modify: `skills/web-automation/scripts/flow.ts`
- Modify: `skills/web-automation/scripts/scrape.ts`
- Modify: `skills/web-automation/scripts/test-minimal.ts`
- Modify: `skills/web-automation/scripts/test-full.ts`
- Modify: `skills/web-automation/scripts/test-profile.ts`
**Step 1: Keep interfaces stable**
- preserve CLI usage where possible
- use CloakBrowser through shared launcher code
- keep one-shot extraction JSON output unchanged except for backend wording if needed
**Step 2: Add compatibility aliases**
- support old `CAMOUFOX_*` env vars where practical
- document new canonical naming
### Task 4: Verify behavior
**Files:**
- Test: `skills/web-automation/scripts/*`
**Step 1: Extractor error checks**
Run:
```bash
cd /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills/.worktrees/web-automation-cloakbrowser && node skills/web-automation/scripts/extract.js
```
Expected: JSON error for missing URL
Run:
```bash
cd /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills/.worktrees/web-automation-cloakbrowser && node skills/web-automation/scripts/extract.js ftp://example.com
```
Expected: JSON error for invalid scheme
**Step 2: Extractor smoke test**
Run:
```bash
cd /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills/.worktrees/web-automation-cloakbrowser && node skills/web-automation/scripts/extract.js https://example.com
```
Expected: JSON result with title `Example Domain` and status `200`
**Step 3: Stateful path verification**
- run one direct `browse.ts`, `scrape.ts`, or `flow.ts` command using the CloakBrowser backend
- confirm persistent-context code still initializes successfully
### Task 5: Update docs and publish
**Files:**
- Modify: `skills/web-automation/SKILL.md`
- Modify: `docs/web-automation.md`
- Modify: `README.md`
- Modify: `docs/README.md`
**Step 1: Update docs**
- replace Camoufox wording with CloakBrowser wording
- replace old setup/install steps
- document any compatibility env vars and new canonical names
**Step 2: Commit and push**
Run:
```bash
git add skills/web-automation docs README.md
git commit -m "refactor: migrate web-automation to cloakbrowser"
git push -u origin feature/web-automation-cloakbrowser
```
**Step 3: Merge and cleanup**
- fast-forward or merge to `main`
- push `main`
- remove the worktree
- delete the feature branch
@@ -0,0 +1,76 @@
# NordVPN Client Docs Refresh Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Refresh the `nordvpn-client` documentation so operators and the OpenClaw agent have complete, accurate setup and troubleshooting guidance for the current macOS and Linux backends.
**Architecture:** Expand the canonical repo doc into a full operator guide, tighten the agent-facing `SKILL.md` to match the current behavior, and lightly update summary docs only if their current one-line descriptions are materially incomplete. Sync the updated `SKILL.md` into the installed OpenClaw workspace copy so runtime guidance matches the repo.
**Tech Stack:** Markdown docs, local repo skill docs, OpenClaw workspace skill sync
---
### Task 1: Refresh canonical operator documentation
**Files:**
- Modify: `docs/nordvpn-client.md`
**Step 1: Rewrite the doc structure**
- Add sections for overview, platform backends, prerequisites, credential paths, install/bootstrap, macOS sudoers setup, command flows, output model, and troubleshooting.
**Step 2: Add exact operator setup details**
- Include the exact `visudo` entry for the helper script.
- Document default token/password file locations.
- Document Homebrew install commands for macOS tooling.
**Step 3: Add safe troubleshooting guidance**
- Include only safe operator procedures from the debugging work:
- invalid token handling
- `sudoReady: false`
- Tailscale suspend/resume expectations
- what normal redacted output includes
- how to use `--debug` when deeper inspection is needed
### Task 2: Refresh agent-facing skill documentation
**Files:**
- Modify: `skills/nordvpn-client/SKILL.md`
- Sync: `/Users/stefano/.openclaw/workspace/skills/nordvpn-client/SKILL.md`
**Step 1: Tighten the skill instructions**
- Keep the doc shorter than the canonical operator guide.
- Ensure it explicitly covers the default credential paths, macOS sudoers requirement, Tailscale suspend/resume behavior, and `--debug` usage.
**Step 2: Sync installed OpenClaw copy**
- Copy the updated repo `SKILL.md` into the installed workspace skill path.
### Task 3: Update summary docs if needed
**Files:**
- Check: `README.md`
- Check: `docs/README.md`
- Modify only if current summary text is materially missing the current backend model.
**Step 1: Review summary descriptions**
- Confirm whether the one-line descriptions already adequately describe Linux CLI + macOS NordLynx/WireGuard.
**Step 2: Update only if necessary**
- Avoid churn if the existing summaries are already sufficient.
### Task 4: Verify and publish
**Files:**
- Verify: `docs/nordvpn-client.md`
- Verify: `skills/nordvpn-client/SKILL.md`
- Verify: `/Users/stefano/.openclaw/workspace/skills/nordvpn-client/SKILL.md`
**Step 1: Run doc verification checks**
- Run: `rg -n "sudoers|visudo|--debug|Tailscale|token.txt|wireguard-helper" docs/nordvpn-client.md skills/nordvpn-client/SKILL.md`
- Expected: all required topics present
**Step 2: Confirm installed workspace skill matches repo skill**
- Run: `cmp skills/nordvpn-client/SKILL.md /Users/stefano/.openclaw/workspace/skills/nordvpn-client/SKILL.md`
- Expected: no output
**Step 3: Commit and push**
- Commit message: `docs: expand nordvpn client setup and troubleshooting`
@@ -0,0 +1,40 @@
# NordVPN macOS DNS Design
## Goal
Keep NordVPN DNS while connected on macOS, but only apply it to active physical services so the WireGuard backend does not break Tailscale or other virtual interfaces.
## Behavior
- Keep the generated WireGuard config free of `DNS = ...`
- During `connect` on macOS:
- detect active physical network services
- snapshot current DNS/search-domain settings
- set NordVPN DNS only on those physical services
- During `disconnect`:
- restore the saved DNS/search-domain settings
- During failed `connect` after DNS changes:
- restore DNS before returning the error
## DNS Values
- IPv4 primary: `103.86.96.100`
- IPv4 secondary: `103.86.99.100`
- No IPv6 DNS for now
## Service Selection
Include only enabled physical services from `networksetup`.
Exclude names matching:
- Tailscale
- Bridge
- Thunderbolt Bridge
- Loopback
- VPN
- utun
## Persistence
- Save DNS snapshot under `~/.nordvpn-client`
- Overwrite on each successful connect
- Clear after successful disconnect restore
## Verification
- Unit tests for service selection and DNS snapshot/restore helpers
- Direct logic/config tests
- Avoid live connect tests from this session unless explicitly requested because they can drop connectivity
@@ -0,0 +1,11 @@
# NordVPN macOS DNS Plan
1. Add macOS DNS state file support under `~/.nordvpn-client`.
2. Implement helpers to enumerate eligible physical services and snapshot existing DNS/search-domain settings.
3. Implement helpers to apply NordVPN DNS only to eligible physical services.
4. Implement helpers to restore previous DNS/search-domain settings on disconnect or failed connect.
5. Add unit tests for service filtering and DNS state transitions.
6. Update skill/docs to explain macOS physical-service DNS management.
7. Sync the installed workspace copy.
8. Run tests and non-destructive verification.
9. Commit and push.
@@ -0,0 +1,26 @@
# NordVPN Tailscale Coordination Design
## Goal
Stabilize macOS NordVPN connects by explicitly stopping Tailscale before bringing up the NordVPN WireGuard tunnel, then restarting Tailscale after NordVPN disconnects.
## Behavior
- macOS only
- on `connect`:
- detect whether Tailscale is active
- if active, stop it and record that state
- bring up NordVPN
- on `disconnect`:
- tear down NordVPN
- if the skill stopped Tailscale earlier, start it again
- clear the saved state
- on connect failure after stopping Tailscale:
- attempt to start Tailscale again before returning the error
## State
- persist `tailscaleWasActive` under `~/.nordvpn-client`
- only restart Tailscale if the skill actually stopped it
## Rollback target if successful
- remove the temporary macOS physical-service DNS management patch
- restore the simpler NordVPN config path that uses NordVPN DNS directly in the WireGuard config
- keep Tailscale suspend/resume as the macOS coexistence solution
@@ -0,0 +1,10 @@
# NordVPN Tailscale Coordination Plan
1. Add macOS Tailscale state file support under `~/.nordvpn-client`.
2. Implement helpers to detect, stop, and start Tailscale on macOS.
3. Add unit tests for Tailscale state transitions.
4. Wire Tailscale stop into macOS `connect` before WireGuard up.
5. Wire Tailscale restart into macOS `disconnect` and connect-failure rollback.
6. Sync the installed workspace copy.
7. Run tests and non-destructive verification.
8. Commit and push.
@@ -0,0 +1,180 @@
# Property Assessor WhatsApp-Safe Runtime Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Keep WhatsApp property-assessor runs moving by failing fast on silent discovery/photo hangs, avoiding helper subprocesses during core analysis, and reserving subprocess use to the final PDF render attempt only.
**Architecture:** Add small timeout guards around the existing in-process listing discovery and photo extraction calls so one quiet browser-backed task cannot stall the entire assessment. Tighten the live skill and published docs so messaging runs treat chat-native source collection as the default path and helper commands as non-chat or final-render-only tools.
**Tech Stack:** TypeScript, Node test runner, existing OpenClaw property-assessor skill, existing OpenClaw web-automation modules, Markdown docs.
---
### Task 1: Add failing timeout tests for discovery and photo extraction
**Files:**
- Modify: `skills/property-assessor/tests/assessment.test.ts`
**Step 1: Write the failing test**
Add tests that stub discovery/photo functions with never-resolving promises and assert that:
- listing discovery returns `null` URLs and records timeout attempts
- photo extraction returns `not completed` instead of hanging forever
**Step 2: Run test to verify it fails**
Run: `npm test -- --test-name-pattern "times out"`
Expected: FAIL because current code never times out or records timeout attempts.
**Step 3: Write minimal implementation**
Implement the smallest timeout wrapper needed for the tests to pass.
**Step 4: Run test to verify it passes**
Run: `npm test -- --test-name-pattern "times out"`
Expected: PASS
**Step 5: Commit**
```bash
git add skills/property-assessor/tests/assessment.test.ts
git commit -m "test: cover stalled discovery and photo extraction"
```
### Task 2: Implement hard timeout guards in the live assessment path
**Files:**
- Create: `skills/property-assessor/src/async-timeout.ts`
- Modify: `skills/property-assessor/src/listing-discovery.ts`
- Modify: `skills/property-assessor/src/photo-review.ts`
**Step 1: Write the failing test**
Use the tests from Task 1 as the red phase.
**Step 2: Run test to verify it fails**
Run: `npm test -- --test-name-pattern "times out"`
Expected: FAIL
**Step 3: Write minimal implementation**
Add:
- a shared timeout helper for async operations
- timeout-wrapped Zillow/HAR discovery in `listing-discovery.ts`
- timeout-wrapped Zillow/HAR photo extraction in `photo-review.ts`
- clear timeout attempt messages so the assessment can continue honestly
**Step 4: Run test to verify it passes**
Run: `npm test -- --test-name-pattern "times out"`
Expected: PASS
**Step 5: Commit**
```bash
git add skills/property-assessor/src/async-timeout.ts skills/property-assessor/src/listing-discovery.ts skills/property-assessor/src/photo-review.ts skills/property-assessor/tests/assessment.test.ts
git commit -m "fix: fail fast on stalled property-assessor extraction steps"
```
### Task 3: Tighten live skill instructions for WhatsApp-safe execution
**Files:**
- Modify: `../skills/property-assessor/SKILL.md`
**Step 1: Write the failing test**
No automated test. Use the documented runtime rule as the spec:
- WhatsApp/messaging runs must avoid helper subprocesses for core analysis
- only the final PDF render attempt may use the helper subprocess path
- `update?` must remain status-only
**Step 2: Run verification to confirm current docs are wrong**
Run: `rg -n "scripts/property-assessor assess|node zillow-photos|node har-photos|Good:" ../skills/property-assessor/SKILL.md`
Expected: current doc still presents helper commands as normal chat-safe core workflow.
**Step 3: Write minimal implementation**
Update the live skill doc to:
- prefer `web_search`, `web_fetch`, and bounded `web-automation` for core assessment
- forbid `scripts/property-assessor assess`, `node zillow-photos.js`, `node har-photos.js`, and ad hoc `curl` as the default WhatsApp core path
- allow a single final PDF render attempt only after a decision-grade verdict exists
**Step 4: Run verification**
Run: `sed -n '1,220p' ../skills/property-assessor/SKILL.md`
Expected: the WhatsApp-safe runtime rules are explicit and unambiguous.
**Step 5: Commit**
```bash
git add ../skills/property-assessor/SKILL.md
git commit -m "docs: clarify whatsapp-safe property-assessor execution"
```
### Task 4: Mirror the runtime guidance into the published repo docs
**Files:**
- Modify: `docs/property-assessor.md`
- Modify: `docs/web-automation.md`
**Step 1: Write the failing test**
No automated test. The spec is consistency with the live skill instructions.
**Step 2: Run verification to confirm current docs drift**
Run: `rg -n "node zillow-photos|node har-photos|assess --address" docs/property-assessor.md docs/web-automation.md`
Expected: current docs still imply subprocess-heavy commands are the standard chat path.
**Step 3: Write minimal implementation**
Document:
- chat-native assessment first
- timeout-protected discovery/photo extraction behavior
- final-render-only subprocess attempt from messaging runs
**Step 4: Run verification**
Run: `sed -n '1,220p' docs/property-assessor.md && sed -n '1,220p' docs/web-automation.md`
Expected: published docs match the live skill behavior.
**Step 5: Commit**
```bash
git add docs/property-assessor.md docs/web-automation.md
git commit -m "docs: document whatsapp-safe property assessment flow"
```
### Task 5: Verify the focused runtime behavior
**Files:**
- Modify: `skills/property-assessor/tests/assessment.test.ts`
- Verify: `skills/property-assessor/src/*.ts`
- Verify: `../skills/property-assessor/SKILL.md`
- Verify: `docs/property-assessor.md`
- Verify: `docs/web-automation.md`
**Step 1: Run focused tests**
Run: `npm test`
Expected: all `property-assessor` tests pass, including timeout coverage.
**Step 2: Run targeted source verification**
Run: `rg -n "withTimeout|timed out|final PDF render" skills/property-assessor/src ../skills/property-assessor/SKILL.md docs/property-assessor.md docs/web-automation.md`
Expected: timeout guards and the final-render-only messaging rule are present.
**Step 3: Inspect git status**
Run: `git status --short`
Expected: only intended files are modified.
**Step 4: Commit**
```bash
git add skills/property-assessor/src/async-timeout.ts skills/property-assessor/src/listing-discovery.ts skills/property-assessor/src/photo-review.ts skills/property-assessor/tests/assessment.test.ts ../skills/property-assessor/SKILL.md docs/property-assessor.md docs/web-automation.md docs/plans/2026-03-28-property-assessor-whatsapp-safe-runtime.md
git commit -m "fix: make property-assessor safer for whatsapp runs"
```
+477
View File
@@ -0,0 +1,477 @@
# property-assessor
Decision-grade residential property assessment skill for OpenClaw, with official public-record enrichment and fixed-template PDF report rendering.
## Overview
`property-assessor` is for evaluating a condo, townhouse, house, or similar residential property from an address or listing URL and ending with a practical recommendation such as `buy`, `pass`, or `only below X`.
If the subject property has an apartment / unit / suite number, include it. Discovery is now unit-aware for Zillow and HAR when unit data is present, while still supporting plain single-family addresses that have no unit.
The skill is intended to:
- normalize the property across listing sources
- review listing photos before making condition claims
- incorporate official public-record / appraisal-district context when available
- compare the property against comps and carrying costs
- produce a fixed-format PDF report, not just ad hoc chat prose
## Standalone helper usage
This skill now ships with a small TypeScript helper package for three tasks:
- assembling an address-first preliminary assessment payload
- locating official public-record jurisdiction from an address
- rendering a fixed-template PDF report
From `skills/property-assessor/`:
```bash
npm install
scripts/property-assessor --help
```
The wrapper script uses the skill-local Node dependencies under `node_modules/`.
Delivery rule:
- If the user explicitly says to email or send the finished PDF to a stated target address, that counts as delivery authorization once the report is ready.
- The agent should not ask for a second `send it` confirmation unless the user changed the destination or showed uncertainty.
- Final property-assessor delivery should be sent as Luke via the Luke Google Workspace wrapper, while the destination remains whatever address the user specified.
## Commands
```bash
scripts/property-assessor assess --address "4141 Whiteley Dr, Corpus Christi, TX 78418" --assessment-purpose "investment property"
scripts/property-assessor assess --address "4141 Whiteley Dr, Corpus Christi, TX 78418" --assessment-purpose "investment property" --recipient-email "buyer@example.com"
scripts/property-assessor assess --address "4141 Whiteley Dr, Corpus Christi, TX 78418" --assessment-purpose "investment property" --recipient-email "buyer@example.com" --output /tmp/property-assessment.pdf
scripts/property-assessor locate-public-records --address "4141 Whiteley Dr, Corpus Christi, TX 78418"
scripts/property-assessor locate-public-records --address "4141 Whiteley Dr, Corpus Christi, TX 78418" --parcel-id "14069438"
scripts/property-assessor locate-public-records --address "4141 Whiteley Dr, Corpus Christi, TX 78418" --listing-geo-id "233290"
scripts/property-assessor render-report --input examples/report-payload.example.json --output /tmp/property-assessment.pdf
```
## Core workflow
Default operating sequence:
1. Normalize the address and property type.
2. Resolve public-record jurisdiction from the address.
3. Discover accessible listing sources for the same property.
4. Build a baseline fact set.
5. Review listing photos before making condition claims.
6. Pull same-building or nearby comps.
7. Underwrite carry costs and risk factors.
8. Render the final report as a fixed-template PDF.
Operational rule:
- The helper returning a preliminary payload is not the end of the job.
- For a user request that clearly asks for a full assessment or PDF delivery, the agent is expected to continue the missing analysis after the helper returns.
- Preliminary helper output should be treated as structured scaffolding for the remaining work, not as a reason to stop and wait for another user nudge.
- In chat/messaging runs, do not waste the turn on `npm install` or `npm ci` when the local skill dependencies are already present.
- If the user asks `update?` or `and?` mid-run, treat that as a status request and continue the same assessment rather than restarting or stopping at the last checkpoint.
- In WhatsApp or similar messaging runs, keep the core analysis on `web-automation` plus `web_fetch`. Treat `web_search` as a narrow fallback for alternate-URL discovery only, not the default path for Zillow/HAR/CAD work.
- Do not start Zillow/HAR property discovery or photo review from Brave-backed `web_search` when `web-automation` can open the candidate listing source directly.
- For CAD/public-record lookup, prefer official assessor/CAD pages via `web_fetch` first and `web-automation` second if the site needs rendered interaction.
- In Texas runs, do not use `https://www.texas.gov/propertytaxes/search/` as the CAD lookup path; use the address-first CAD/helper path or the discovered county CAD pages directly.
- In those messaging runs, reserve subprocess use for a single final `render-report` attempt after the verdict and fair-value range are complete.
- In those messaging runs, do not start Gmail/email-send skill discovery or delivery tooling until the report content is complete and the PDF is ready to render or already rendered.
- Property-assessor delivery emails should be sent as Luke from Luke's Google Workspace account, while still delivering to the user-specified destination.
- Use `zsh ~/.openclaw/workspace/bin/gog-luke gmail send --to "<target>" --subject "<subject>" --body "<body>" --attach "<pdf-path>"` for final PDF delivery.
- Do not route property-assessor delivery through generic `gog` or the Stefano helper `node ~/.openclaw/workspace/integrations/google-workspace/gw.js`.
- If the agent needs to confirm Luke auth before sending, use `zsh ~/.openclaw/workspace/bin/gog-luke auth list --check --plain`.
- Treat a silent helper as a failed helper in messaging runs. If a helper produces no useful output within a short bound, abandon it and continue with the chat-native path instead of repeatedly polling it.
- If Zillow photo extraction fails, immediately continue with HAR photo fallback or the next available rendered listing/photo source rather than stopping the assessment.
- After a Zillow/HAR photo miss, continue the comp and CAD/public-record work in the same run. A photo-source miss is a fallback event, not a terminal state.
- If the original request already authorized sending the finished PDF to a stated email address, do not pause for a redundant send-confirmation prompt after rendering.
- If final PDF render/send fails, return the completed decision-grade report in chat and report delivery failure separately rather than restarting the whole assessment.
### `assess`
```bash
scripts/property-assessor assess --address "<street-address>" --assessment-purpose "<purpose>"
scripts/property-assessor assess --address "<street-address>" --assessment-purpose "<purpose>" --recipient-email "<target@example.com>"
```
Current behavior:
- starts from the address
- requires an assessment purpose for decision-grade analysis
- does not assume the assessment purpose from prior thread context unless the user explicitly says the purpose is unchanged
- automatically runs public-record / appraisal-district lookup
- keeps CAD-site selection address-driven and jurisdiction-specific; it does not hardcode one county's CAD as the global source
- when a supported official CAD detail host is found, captures direct property facts such as property ID/account, owner, legal description, assessed value, exemptions, and the official property-detail URL
- automatically tries to discover Zillow and HAR listing URLs from the address when no listing URL is provided
- starts Zillow and HAR listing discovery in parallel so HAR can already be in flight if Zillow misses or stalls
- runs Zillow photo extraction first, then HAR as fallback when available
- gives Zillow a longer source-specific discovery/photo window than the generic fallback path, because some exact-unit Zillow pages resolve more slowly than HAR or public-record lookups
- reuses the OpenClaw web-automation logic in-process instead of spawning nested helper commands
- fails fast when Zillow/HAR discovery or photo extraction stalls instead of hanging indefinitely
- returns a structured preliminary report payload
- does not require recipient email(s) for the analysis-only run
- asks for recipient email(s) only when PDF rendering is explicitly requested
- does not render/send the PDF from a preliminary helper payload with `decision: pending`
- does not render/send the PDF when `photoReview.status` is not `completed`
- only renders the fixed-template PDF after a decision-grade verdict and fair-value range are actually present
Expected agent behavior:
- if the user asked for the full assessment, continue beyond the preliminary helper output
- fill the remaining gaps with listing facts, comp work, condition interpretation, and valuation logic
- require completed subject-unit photo review before treating the report as decision-grade enough for PDF delivery
- only stop early when there is a real blocker, not merely because the helper stopped at a checkpoint
Important limitation:
- this implementation now wires the address-first intake, purpose-aware framing, public-record lookup, listing discovery, and photo-source extraction
- it still does not perform full comp analysis, pricing judgment, or completed carry underwriting inside the helper itself
- those deeper decision steps are still governed by the skill workflow after the helper assembles the enriched payload
## Source priority
Unless the user says otherwise, preferred listing/source order is:
1. Zillow
2. Redfin
3. Realtor.com
4. HAR / Homes.com / brokerage mirrors
5. county or appraisal pages
Public-record / assessor data should be linked in the final result when available.
## Public-record enrichment
The skill should not rely on listing-site geo IDs as if they were assessor record identifiers.
Correct approach:
1. start from the street address
2. resolve the address to state/county/FIPS/GEOID
3. identify the official public-record jurisdiction
4. use parcel/APN/account identifiers when available
5. link the official jurisdiction page and any direct property page used
### `locate-public-records`
```bash
scripts/property-assessor locate-public-records --address "<street-address>"
```
Current behavior:
- uses the official Census geocoder
- when Census address matching misses, falls back to an external address geocoder and then resolves official Census geographies from coordinates
- retries fallback geocoding without the unit suffix when a condo/unit-qualified address is too specific for the fallback provider
- returns matched address, county/state/FIPS, and block GEOID context
- for Texas, returns:
- Texas Comptroller county directory page
- appraisal district contact/site details
- tax assessor/collector contact/site details
- official CAD property-detail facts when a supported county adapter can retrieve them from the discovered CAD site
Important rules:
- Zillow/Redfin/HAR geo IDs are hints only
- parcel/APN/account IDs are stronger search keys than listing geo IDs
- if Zillow exposes a parcel/APN/account number on the listing, capture it and use that identifier in CAD lookup before falling back to address-only matching
- official jurisdiction pages should be linked in the final report
- if a direct property detail page is accessible, its data should be labeled as official public-record evidence
### Texas support
Texas is the first-class public-record path in this implementation.
For Texas addresses, the helper resolves:
- the official Census geocoder link
- the official Texas Comptroller county directory page
- the appraisal district website
- the tax assessor/collector website
- the official CAD property-detail page when a supported adapter can identify and retrieve the subject record
That output should be used by the skill to:
- identify the correct CAD
- attempt address / parcel / account lookup on the discovered CAD site for that county
- prefer Zillow-exposed parcel/APN/account identifiers over address-only search when the listing provides them
- capture official owner / legal / assessed-value evidence when a public detail page is available
- treat county-specific CAD detail retrieval as an adapter layer on top of generic county/jurisdiction resolution
Nueces-specific note:
- when using Nueces CAD `By ID` / `Geographic ID`, insert a dash after the first 4 digits and again after the first 8 digits, for example `123456789012` -> `1234-5678-9012`
Recommended fields to capture from official records when accessible:
- account number
- owner name
- land value
- improvement value
- assessed total
- exemptions
- official property-detail URL
## Minimum data to capture
For the subject property, capture when available:
- address
- list price or last known list price
- property type
- beds / baths
- square footage
- lot size if relevant
- year built
- HOA fee and included services
- taxes
- days on market
- price history
- parking
- waterfront / flood clues
- subdivision or building name
- same-building or nearby active inventory
- listing photos and visible condition cues
- public-record jurisdiction and linked official source
- account / parcel / tax ID if confirmed
- official assessed values and exemptions if confirmed
## Photo-review rules
Photo review is mandatory when photos are exposed by a listing source.
Do not make strong condition claims from structured text alone if photos are available.
Preferred photo-access order:
1. Zillow extractor
2. HAR extractor
3. Realtor.com photo page
4. brokerage mirror or other accessible listing mirror
Use the dedicated `web-automation` extractors first:
```bash
cd ~/.openclaw/workspace/skills/web-automation/scripts
node zillow-photos.js "<zillow-listing-url>"
node har-photos.js "<har-listing-url>"
```
When those extractors return `imageUrls`, that returned set is the photo-review set.
## Approval-safe command shape
For local/manual runs, prefer file-based commands.
For chat-driven messaging runs, prefer native `web_search`, `web_fetch`, and bounded browser actions first. Treat these commands as local/manual helpers or non-chat fallbacks, not as the default core-analysis path.
Good:
- `scripts/property-assessor assess --address "..." --assessment-purpose "..."`
- `node check-install.js`
- `node zillow-discover.js "<street-address>"`
- `node har-discover.js "<street-address>"`
- `node zillow-photos.js "<url>"`
- `node har-photos.js "<url>"`
- `scripts/property-assessor locate-public-records --address "..."`
- `scripts/property-assessor render-report --input ... --output ...`
- `web_fetch` to read an official CAD / assessor page when the TypeScript helper needs a plain page fetch
Messaging default:
- `web_search` to discover listing and public-record URLs
- `web_fetch` for official CAD / assessor pages and accessible listing pages
- bounded `web-automation` actions for rendered listing/photo views
- one final `scripts/property-assessor render-report ...` attempt only after the decision-grade report is complete
Avoid when possible:
- `node -e "..."`
- `node --input-type=module -e "..."`
- `python3 - <<'PY' ... PY`
- `python -c "..."`
- raw `bash -lc '...'` or `zsh -lc '...'` probes for CAD / public-record lookup
Reason:
- OpenClaw exec approvals are path-based, and inline shell / interpreter forms are treated conservatively in allowlist mode.
- For `property-assessor`, CAD and public-record lookup should stay on the skills file-based TypeScript helper path or use `web_fetch`.
- If the workflow drifts into an ad hoc shell snippet, that is not the approved skill path and can still trigger Control UI approval prompts.
## PDF report template
The final deliverable should be a fixed-template PDF, not a one-off layout.
Template reference:
- `skills/property-assessor/references/report-template.md`
Current renderer:
```bash
scripts/property-assessor assess --address "<street-address>" --recipient-email "<target@example.com>"
scripts/property-assessor render-report --input "<report-payload-json>" --output "<output-pdf>"
```
The fixed template includes:
1. Report header
2. Verdict panel
3. Subject-property summary table
4. Snapshot
5. What I Like
6. What I Do Not Like
7. Comp View
8. Underwriting / Carry View
9. Risks and Diligence Items
10. Photo Review
11. Public Records
12. Source Links
13. Notes page
### Recipient email gate
The report must not be rendered or sent unless target recipient email address(es) are known.
This requirement applies when the operator is actually rendering or sending the PDF.
It should not interrupt a normal analysis-only `assess` run.
If the prompt does not include recipient email(s), the skill should:
- stop
- ask for target recipient email address(es)
- not finalize the PDF workflow yet
The renderer enforces this. If `recipientEmails` is missing or empty, it fails with:
`Missing target email. Stop and ask the user for target email address(es) before generating or sending the property assessment PDF.`
## Example payload
Sample payload:
- `skills/property-assessor/examples/report-payload.example.json`
This is the easiest way to test the renderer without building a report payload from scratch.
## Output contract
The assessment itself should remain concise but decision-grade.
Recommended narrative structure:
1. Snapshot
2. What I like
3. What I do not like
4. Comp view
5. Underwriting / carry view
6. Risks and diligence items
7. Verdict with fair-value range and offer guidance
It must also explicitly include:
- `Photo source attempts: ...`
- `Photo review: completed via <source>` or `Photo review: not completed`
- public-record / CAD evidence and links when available
## Validation flow
### 1. Install the helper package locally
```bash
cd ~/.openclaw/workspace/skills/property-assessor
npm install
npm test
```
### 2. Run address-first assess without recipient email
```bash
cd ~/.openclaw/workspace/skills/property-assessor
scripts/property-assessor assess --address "4141 Whiteley Dr, Corpus Christi, TX 78418" --assessment-purpose "investment property"
```
Expected shape:
- `needsAssessmentPurpose: false`
- `needsRecipientEmails: false`
- public-record / CAD jurisdiction included in the returned payload
- `photoReview.imageUrls` populated when Zillow or HAR extraction succeeds
- no PDF generated yet
- explicit message saying the payload is ready and email is only needed when rendering or sending the PDF
### 3. Run public-record lookup directly
```bash
cd ~/.openclaw/workspace/skills/property-assessor
scripts/property-assessor locate-public-records --address "4141 Whiteley Dr, Corpus Christi, TX 78418"
```
Expected shape:
- state/county/FIPS/GEOID present
- official Census geocoder link present
- for Texas: Comptroller county directory link present
- for Texas: appraisal district and tax assessor/collector contacts present
### 4. Run assess with recipient email and render the PDF
```bash
cd ~/.openclaw/workspace/skills/property-assessor
scripts/property-assessor assess --address "4141 Whiteley Dr, Corpus Christi, TX 78418" --assessment-purpose "investment property" --recipient-email "buyer@example.com" --output /tmp/property-assessment.pdf
```
Expected result:
- `needsRecipientEmails: false`
- JSON success payload with `outputPath`
- a non-empty PDF written to `/tmp/property-assessment.pdf`
### 5. Run PDF render with the sample payload
```bash
cd ~/.openclaw/workspace/skills/property-assessor
scripts/property-assessor render-report --input examples/report-payload.example.json --output /tmp/property-assessment.pdf
```
Expected result:
- JSON success payload with `outputPath`
- a non-empty PDF written to `/tmp/property-assessment.pdf`
### 6. Verify the email gate
Run the renderer with a payload that omits `recipientEmails`.
Expected result:
- non-zero exit
- explicit message telling the operator to stop and ask for target recipient email(s)
### 7. Verify the end-to-end skill behavior
When testing `property-assessor` itself, confirm the assessment:
- starts from the address when available
- uses Zillow first for photo extraction, HAR as fallback
- frames the analysis around the stated assessment purpose
- uses official public-record jurisdiction links when available
- does not treat listing geo IDs as assessor keys
- asks for assessment purpose if it was not provided
- asks for recipient email(s) if they were not provided
- renders the final report through the fixed PDF template once recipient email(s) are known
## Related files
- skill instructions:
- `skills/property-assessor/SKILL.md`
- underwriting heuristics:
- `skills/property-assessor/references/underwriting-rules.md`
- PDF template rules:
- `skills/property-assessor/references/report-template.md`
- sample report payload:
- `skills/property-assessor/examples/report-payload.example.json`
- photo extraction docs:
- `docs/web-automation.md`
+103
View File
@@ -0,0 +1,103 @@
# Spotify
The Spotify skill adds a local helper for Spotify Web API playlist work from OpenClaw.
## Scope
- search Spotify tracks
- list the current user's playlists
- create private or public playlists
- add and remove track URIs
- search and add tracks
- import tracks from text lists, M3U/M3U8 playlists, and local folders
The skill uses OAuth2 Authorization Code with PKCE. It does not need a Spotify client secret and does not use browser automation for Spotify operations.
## Setup
Create the local credential directory:
```bash
mkdir -p ~/.openclaw/workspace/.clawdbot/credentials/spotify
chmod 700 ~/.openclaw/workspace/.clawdbot/credentials/spotify
$EDITOR ~/.openclaw/workspace/.clawdbot/credentials/spotify/config.json
```
Example `config.json`:
```json
{
"clientId": "your-spotify-client-id",
"redirectUri": "http://127.0.0.1:8888/callback"
}
```
Run auth from the active OpenClaw skill copy:
```bash
cd ~/.openclaw/workspace/skills/spotify
scripts/setup.sh
```
Or run only the OAuth login after dependencies are installed:
```bash
scripts/spotify auth
scripts/spotify status --json
```
Tokens are written to the local credentials directory as `token.json` with owner-only file mode when the filesystem supports it. Do not print token files.
## Commands
```bash
scripts/spotify status --json
scripts/spotify search "Radiohead Karma Police" --limit 3 --json
scripts/spotify list-playlists --limit 10 --json
scripts/spotify create-playlist "OpenClaw Mix" --description "Created by OpenClaw" --json
scripts/spotify add-to-playlist "<playlistId>" "spotify:track:..." --json
scripts/spotify remove-from-playlist "<playlistId>" "spotify:track:..." --json
scripts/spotify search-and-add "<playlistId>" "Radiohead Karma Police" --json
scripts/spotify import "/path/to/tracks.txt" --playlist "Imported Mix" --json
scripts/spotify import "/path/to/playlist.m3u8" --playlist-id "<playlistId>" --json
scripts/spotify import "/path/to/music-folder" --playlist "Folder Import" --json
```
`--playlist NAME` always creates a new playlist, private by default unless `--public` is provided. Spotify allows duplicate playlist names, so use `--playlist-id ID` when updating an existing playlist.
## Import Behavior
Text imports ignore blank lines and comment lines starting with `#` or `//`.
M3U/M3U8 imports use `#EXTINF` metadata when present and fall back to the filename otherwise.
Folder imports recursively scan supported audio filenames and ignore non-audio files.
The importer searches Spotify once per parsed candidate, adds the first match, reports misses, and skips duplicate Spotify URI matches.
## Endpoint Notes
This skill uses the current Spotify playlist endpoints:
```text
GET /v1/me
GET /v1/search?type=track&q=<query>&limit=<1-10>
GET /v1/me/playlists?limit=<n>&offset=<n>
POST /v1/me/playlists
POST /v1/playlists/{id}/items
DELETE /v1/playlists/{id}/items
POST https://accounts.spotify.com/api/token
```
Do not use the removed 2026 endpoints:
```text
POST /v1/users/{user_id}/playlists
GET /v1/users/{id}/playlists
POST /v1/playlists/{id}/tracks
DELETE /v1/playlists/{id}/tracks
```
## Live Smoke Caution
Spotify does not offer a normal delete-playlist Web API operation. Any live smoke that creates a playlist must be explicitly approved because the playlist can only be manually cleaned up later.
+302
View File
@@ -0,0 +1,302 @@
# us-cpa
`us-cpa` is a Python CLI plus OpenClaw skill wrapper for U.S. federal individual tax work.
## Standalone package usage
From `skills/us-cpa/`:
```bash
python3 -m ensurepip --upgrade
python3 -m pip install --upgrade pip setuptools wheel
python3 -m pip install -e '.[dev]'
us-cpa --help
```
Without installing, the repo-local wrapper works directly:
```bash
skills/us-cpa/scripts/us-cpa --help
```
## OpenClaw installation
To install the skill for OpenClaw itself, copy the repo skill into the workspace skill directory and install its Python dependencies there.
1. Sync the repo copy into the workspace:
```bash
rsync -a --delete --exclude '.venv' \
~/.openclaw/workspace/projects/stef-openclaw-skills/skills/us-cpa/ \
~/.openclaw/workspace/skills/us-cpa/
```
2. Create a workspace-local virtualenv and install the package:
```bash
cd ~/.openclaw/workspace/skills/us-cpa
python3 -m venv .venv
. .venv/bin/activate
python3 -m ensurepip --upgrade
python3 -m pip install --upgrade pip setuptools wheel
python3 -m pip install -e '.[dev]'
```
3. Verify the installed workspace wrapper:
```bash
~/.openclaw/workspace/skills/us-cpa/scripts/us-cpa --help
```
The wrapper prefers `.venv/bin/python` inside the skill directory when present, so OpenClaw can run the workspace copy without relying on global Python packages.
Keep the `--exclude '.venv'` flag on future syncs, otherwise `rsync --delete` will remove the workspace virtualenv.
## Current Milestone
Current implementation now includes:
- deterministic cache layout under `~/.cache/us-cpa` by default
- `fetch-year` download flow for the bootstrap IRS corpus
- source manifest with URL, hash, authority rank, and local path traceability
- primary-law URL building for IRC and Treasury regulation escalation
- case-folder intake, document registration, and machine-usable fact extraction from JSON, text, and PDF inputs
- question workflow with conversation and memo output
- prepare workflow for the current supported multi-form 1040 package
- review workflow with findings-first output
- fillable-PDF first rendering with overlay fallback
- e-file-ready draft export payload generation
## CLI Surface
```bash
skills/us-cpa/scripts/us-cpa question --question "What is the standard deduction?" --tax-year 2025
skills/us-cpa/scripts/us-cpa question --question "What is the standard deduction?" --tax-year 2025 --style memo --format markdown
skills/us-cpa/scripts/us-cpa prepare --tax-year 2025 --case-dir ~/tax-cases/2025-jane-doe
skills/us-cpa/scripts/us-cpa review --tax-year 2025 --case-dir ~/tax-cases/2025-jane-doe
skills/us-cpa/scripts/us-cpa fetch-year --tax-year 2025
skills/us-cpa/scripts/us-cpa extract-docs --tax-year 2025 --case-dir ~/tax-cases/2025-jane-doe --create-case --case-label "Jane Doe" --facts-json ./facts.json
skills/us-cpa/scripts/us-cpa render-forms --tax-year 2025 --case-dir ~/tax-cases/2025-jane-doe
skills/us-cpa/scripts/us-cpa export-efile-ready --tax-year 2025 --case-dir ~/tax-cases/2025-jane-doe
```
## Tax-Year Cache
Default cache root:
```text
~/.cache/us-cpa
```
Override for isolated runs:
```bash
US_CPA_CACHE_DIR=/tmp/us-cpa-cache skills/us-cpa/scripts/us-cpa fetch-year --tax-year 2025
```
Current `fetch-year` bootstrap corpus for tax year `2025` is verified against live IRS `irs-prior` PDFs for:
- Form 1040
- Schedules 1, 2, 3, A, B, C, D, E, SE, and 8812
- Forms 8949, 4562, 4797, 6251, 8606, 8863, 8889, 8959, 8960, 8995, 8995-A, 5329, 5695, and 1116
- General Form 1040 instructions and selected schedule/form instructions
Current bundled tax-year computation data:
- 2024
- 2025
Other years fetch/source correctly, but deterministic return calculations currently stop with an explicit unsupported-year error until rate tables are added.
Adding a new supported year is a deliberate data-table change in `tax_years.py`, not an automatic runtime discovery step. That is intentional for tax-engine correctness.
## Interaction Model
- `question`
- stateless by default
- optional case context
- `prepare`
- requires a case directory
- if none exists, OpenClaw should ask whether to create one and where
- `review`
- requires a case directory
- can operate on an existing or newly-created review case
## Planned Case Layout
```text
<case-dir>/
input/
extracted/
return/
output/
reports/
issues/
sources/
```
Current implementation writes:
- `case-manifest.json`
- `extracted/facts.json`
- `issues/open-issues.json`
## Intake Flow
Current `extract-docs` supports:
- `--create-case`
- `--case-label`
- `--facts-json <path>`
- repeated `--input-file <path>`
Behavior:
- creates the full case directory layout when `--create-case` is used
- copies input documents into `input/`
- stores normalized facts with source metadata in `extracted/facts.json`
- extracts machine-usable facts from JSON/text/PDF documents where supported
- appends document registry entries to `case-manifest.json`
- stops with a structured issue and non-zero exit if a new fact conflicts with an existing stored fact
## Output Contract
- JSON by default
- markdown available with `--format markdown`
- `question` supports:
- `--style conversation`
- `--style memo`
- `question` emits answered analysis output
- `prepare` emits a prepared return package summary
- `export-efile-ready` emits a draft e-file-ready payload
- `review` emits a findings-first review result
- `fetch-year` emits a downloaded manifest location and source count
## Question Engine
Current `question` implementation:
- loads the cached tax-year corpus
- searches the downloaded IRS corpus for relevant authorities and excerpts
- returns one canonical analysis object with:
- authorities
- excerpts
- confidence / risk
- primary-law escalation only when the IRS corpus is still insufficient
- renders that analysis as:
- conversational output
- memo output
In OpenClaw, the model should answer the user from the returned IRS excerpts when `primaryLawRequired` is `false`, rather than merely repeating the CLI summary.
## Form Rendering
Current rendering path:
- official IRS PDFs from the cached tax-year corpus
- deterministic field-fill when usable AcroForm fields are present
- overlay rendering onto those official PDFs using `reportlab` + `pypdf` as fallback
- artifact manifest written to `output/artifacts.json`
Current rendered form support:
- field-fill support for known mapped fillable forms
- overlay generation for the current required-form set resolved by the return model
Current review rule:
- field-filled artifacts are not automatically flagged for review
- overlay-rendered artifacts are marked `reviewRequired: true`
Overlay coordinates are currently a fallback heuristic and are not treated as line-perfect authoritative field maps. Overlay output must be visually reviewed before any filing/export handoff.
## Preparation Workflow
Current `prepare` implementation:
- loads case facts from `extracted/facts.json`
- normalizes them into the current supported federal return model
- preserves source provenance for normalized values
- computes the current supported 1040 package
- resolves required forms across the current supported subset
- writes:
- `return/normalized-return.json`
- `output/artifacts.json`
- `reports/prepare-summary.json`
Current supported calculation inputs:
- `filingStatus`
- `spouse.fullName`
- `dependents`
- `wages`
- `taxableInterest`
- `businessIncome`
- `capitalGainLoss`
- `rentalIncome`
- `federalWithholding`
- `itemizedDeductions`
- `hsaContribution`
- `educationCredit`
- `foreignTaxCredit`
- `qualifiedBusinessIncome`
- `traditionalIraBasis`
- `additionalMedicareTax`
- `netInvestmentIncomeTax`
- `alternativeMinimumTax`
- `additionalTaxPenalty`
- `energyCredit`
- `depreciationExpense`
- `section1231GainLoss`
## E-file-ready Export
`export-efile-ready` writes:
- `output/efile-ready.json`
Current export behavior:
- draft-only
- includes required forms
- includes refund or balance due summary
- includes attachment manifest
- includes unresolved issues
## Review Workflow
Current `review` implementation:
- recomputes the return from current case facts
- compares stored normalized return values to recomputed values
- flags source-fact mismatches for key income fields
- flags likely omitted income when document-extracted facts support an amount the stored return omits
- checks whether required rendered artifacts are present
- flags high-complexity forms for specialist follow-up
- flags overlay-rendered artifacts as requiring human review
- sorts findings by severity
Current render modes:
- `--style conversation`
- `--style memo`
## Scope Rules
- U.S. federal individual returns only in v1
- official IRS artifacts are the target output for compiled forms
- conflicting facts must stop the workflow for user resolution
## Authority Ranking
Current authority classes are ranked to preserve source hierarchy:
- IRS forms
- IRS instructions
- IRS publications
- IRS FAQs
- Internal Revenue Code
- Treasury regulations
- other primary authority
Later research and review flows should consume this ranking rather than inventing their own.
+196 -6
View File
@@ -1,6 +1,6 @@
# web-automation
Automated web browsing and scraping using Playwright, with one-shot extraction and broader Camoufox-based automation under a single skill.
Automated web browsing and scraping using Playwright-compatible CloakBrowser, with one-shot extraction and broader persistent automation under a single skill.
## What this skill is for
@@ -15,20 +15,36 @@ Automated web browsing and scraping using Playwright, with one-shot extraction a
- Use `node skills/web-automation/scripts/extract.js "<URL>"` for one-shot extraction from a single URL
- Use `npx tsx scrape.ts ...` for markdown scraping modes
- Use `npx tsx browse.ts ...`, `auth.ts`, or `flow.ts` for interactive or authenticated flows
- Use `node skills/web-automation/scripts/zillow-discover.js "<street-address>"` or `har-discover.js` to resolve a real-estate listing URL from an address
- Use `node skills/web-automation/scripts/zillow-photos.js "<listing-url>"` or `har-photos.js` for real-estate photo extraction before attempting generic gallery automation
Messaging rule:
- For WhatsApp or similar chat-driven runs, prefer native `web_search`, `web_fetch`, and bounded browser actions over shelling out to helper scripts for every core step.
- Treat the dedicated Zillow/HAR scripts as local/manual helpers, regression checks, or non-chat fallbacks.
- If a messaging workflow needs a subprocess at all, reserve it for a single final delivery step rather than the whole assessment.
## Requirements
- Node.js 20+
- `pnpm`
- Network access to download browser binaries
- Network access to download the CloakBrowser binary on first use or via preinstall
## First-time setup
```bash
cd ~/.openclaw/workspace/skills/web-automation/scripts
pnpm install
npx playwright install chromium
npx camoufox-js fetch
npx cloakbrowser install
pnpm approve-builds
pnpm rebuild better-sqlite3 esbuild
```
## Updating CloakBrowser
```bash
cd ~/.openclaw/workspace/skills/web-automation/scripts
pnpm up cloakbrowser playwright-core
npx cloakbrowser install
pnpm approve-builds
pnpm rebuild better-sqlite3 esbuild
```
@@ -48,15 +64,66 @@ pnpm approve-builds
pnpm rebuild better-sqlite3 esbuild
```
Without this, `browse.ts` and `scrape.ts` may fail before launch because the native bindings are missing.
Without this, helper scripts may fail before launch because the native bindings are missing.
## Prerequisite check
Before running automation, verify the local install and CloakBrowser wiring:
```bash
cd ~/.openclaw/workspace/skills/web-automation/scripts
node check-install.js
```
If this fails, stop and fix setup before troubleshooting site automation.
## Exec approvals allowlist
If OpenClaw keeps prompting for approval when running this skill, add a local allowlist for the main agent:
```bash
openclaw approvals allowlist add --agent main "/opt/homebrew/bin/node"
openclaw approvals allowlist add --agent main "/usr/bin/env"
openclaw approvals allowlist add --agent main "~/.openclaw/workspace/skills/web-automation/scripts/*.js"
openclaw approvals allowlist add --agent main "~/.openclaw/workspace/skills/web-automation/scripts/node_modules/.bin/*"
```
Verify with:
```bash
openclaw approvals get
```
Notes:
- If `node` lives somewhere else, replace `/opt/homebrew/bin/node` with the output of `which node`.
- If matching is inconsistent, replace `~/.openclaw/...` with the full absolute path for the machine.
- Keep the allowlist scoped to the main agent unless there is a clear reason to widen it.
- Prefer file-based commands like `node check-install.js`, `node zillow-photos.js ...`, and `node har-photos.js ...` over inline `node -e ...`. Inline interpreter eval is more likely to trigger approval friction.
- The same applies to `zillow-discover.js` and `har-discover.js`: keep discovery file-based, not inline.
## Common commands
```bash
# Install / wiring check
cd ~/.openclaw/workspace/skills/web-automation/scripts
node check-install.js
# One-shot JSON extraction
node skills/web-automation/scripts/extract.js "https://example.com"
# Browse a page
# Zillow listing discovery from address
node skills/web-automation/scripts/zillow-discover.js "4141 Whiteley Dr, Corpus Christi, TX 78418"
# HAR listing discovery from address
node skills/web-automation/scripts/har-discover.js "4141 Whiteley Dr, Corpus Christi, TX 78418"
# Zillow photo extraction
node skills/web-automation/scripts/zillow-photos.js "https://www.zillow.com/homedetails/..."
# HAR photo extraction
node skills/web-automation/scripts/har-photos.js "https://www.har.com/homedetail/..."
# Browse a page with persistent profile
npx tsx browse.ts --url "https://example.com"
# Scrape markdown
@@ -69,6 +136,111 @@ npx tsx auth.ts --url "https://example.com/login"
npx tsx flow.ts --instruction 'go to https://search.fiorinis.com then type "pippo" then press enter then wait 2s'
```
## Real-estate listing discovery and photo extraction
Use the dedicated Zillow and HAR discovery/photo commands before trying a free-form gallery flow.
Discovery is unit-aware when the address includes an apartment / unit / suite identifier, and still supports plain no-unit addresses for single-family homes.
### Zillow discovery
```bash
cd ~/.openclaw/workspace/skills/web-automation/scripts
node zillow-discover.js "4141 Whiteley Dr, Corpus Christi, TX 78418"
```
What it does:
- opens the Zillow address URL with CloakBrowser
- resolves directly to a property page when Zillow supports the address slug
- otherwise looks for a `homedetails` listing link in the rendered page
- returns the discovered listing URL as JSON
- fails fast with a timeout if the browser-backed discovery stalls
Operational note:
- when imported by `property-assessor`, Zillow discovery is allowed a longer source-specific timeout than the generic helper default, because some exact-unit Zillow pages resolve more slowly than the basic search/listing flow
### HAR discovery
```bash
cd ~/.openclaw/workspace/skills/web-automation/scripts
node har-discover.js "4141 Whiteley Dr, Corpus Christi, TX 78418"
```
What it does:
- opens the HAR address search page
- looks for a confident `homedetail` match in rendered results
- returns the discovered listing URL when HAR exposes a strong enough match
- returns `listingUrl: null` when HAR discovery is not confident enough
- fails fast with a timeout if the browser-backed discovery stalls
### Zillow
```bash
cd ~/.openclaw/workspace/skills/web-automation/scripts
node zillow-photos.js "https://www.zillow.com/homedetails/4141-Whiteley-Dr-Corpus-Christi-TX-78418/2103723704_zpid/"
```
What it does:
- opens the listing page with CloakBrowser
- first checks whether the rendered listing shell already exposes a complete photo set in Zillow's embedded `__NEXT_DATA__` payload
- if the visible `See all XX photos` count is missing, still trusts the embedded set when the page metadata confirms the count or when the embedded set is already clearly substantial
- only tries the `See all photos` / `See all X photos` entry point when the initial structured data is incomplete
- returns direct `photos.zillowstatic.com` image URLs as JSON
- fails fast with a timeout if the browser-backed extraction stalls
Operational note:
- when imported by `property-assessor`, Zillow photo extraction is allowed a longer source-specific timeout than the generic helper default, because some exact-unit Zillow listings expose the correct photo set only after a slower render path
Expected success shape:
- `complete: true`
- `expectedPhotoCount` matches `photoCount`
- `imageUrls` contains the listing photo set
### Zillow identifiers
```bash
cd ~/.openclaw/workspace/skills/web-automation/scripts
node zillow-identifiers.js "https://www.zillow.com/homedetails/6702-Everhart-Rd-APT-T106-Corpus-Christi-TX-78413/2067445642_zpid/"
```
What it does:
- opens the Zillow listing shell without forcing the photo workflow
- inspects embedded `__NEXT_DATA__` plus visible listing text
- extracts parcel/APN-style identifiers when Zillow exposes them
- returns those identifiers so `property-assessor` can use them as stronger CAD lookup keys than listing geo IDs
### HAR
```bash
cd ~/.openclaw/workspace/skills/web-automation/scripts
node har-photos.js "https://www.har.com/homedetail/4141-whiteley-dr-corpus-christi-tx-78418/14069438"
```
What it does:
- opens the HAR listing page
- clicks `Show all photos` / `View all photos`
- extracts the direct `pics.harstatic.com` image URLs from the all-photos page
- fails fast with a timeout if the browser-backed extraction stalls
Expected success shape:
- `complete: true`
- `expectedPhotoCount` matches `photoCount`
- `imageUrls` contains the listing photo set
### Test commands
From `skills/web-automation/scripts`:
```bash
node check-install.js
npm run test:photos
node zillow-discover.js "<street-address>"
node har-discover.js "<street-address>"
node zillow-photos.js "<zillow-listing-url>"
node har-photos.js "<har-listing-url>"
```
Use the live Zillow and HAR URLs above for a known-good regression check.
## One-shot extraction (`extract.js`)
Use `extract.js` when the task is just: open one URL, render it, and return structured content.
@@ -104,6 +276,24 @@ USER_AGENT="Mozilla/5.0 ..." node skills/web-automation/scripts/extract.js "http
- optional `screenshot`
- optional `htmlFile`
## Persistent browsing profile
`browse.ts`, `auth.ts`, `flow.ts`, and `scrape.ts` use a persistent CloakBrowser profile so sessions survive across runs.
Canonical env vars:
- `CLOAKBROWSER_PROFILE_PATH`
- `CLOAKBROWSER_HEADLESS`
- `CLOAKBROWSER_USERNAME`
- `CLOAKBROWSER_PASSWORD`
Legacy aliases still supported for compatibility:
- `CAMOUFOX_PROFILE_PATH`
- `CAMOUFOX_HEADLESS`
- `CAMOUFOX_USERNAME`
- `CAMOUFOX_PASSWORD`
## Natural-language flow runner (`flow.ts`)
Use `flow.ts` when you want a general command style like:
+92 -7
View File
@@ -3,7 +3,7 @@
Google Workspace helper CLI
Commands:
whoami
send --to <email> --subject <text> --body <text> [--html]
send --to <email> --subject <text> --body <text> [--html] [--attach <file>]
search-mail --query <gmail query> [--max 10]
search-calendar --query <text> [--max 10] [--timeMin ISO] [--timeMax ISO] [--calendar primary]
create-event --summary <text> --start <ISO> --end <ISO> [--timeZone America/Chicago] [--description <text>] [--location <text>] [--calendar primary]
@@ -11,7 +11,6 @@
const fs = require('fs');
const path = require('path');
const { google } = require('googleapis');
const DEFAULT_SUBJECT = process.env.GW_IMPERSONATE || 'stefano@fiorinis.com';
const DEFAULT_KEY_CANDIDATES = [
@@ -44,8 +43,12 @@ function parseArgs(argv) {
const next = argv[i + 1];
if (!next || next.startsWith('--')) {
out[key] = true;
} else {
if (Object.hasOwn(out, key)) {
out[key] = Array.isArray(out[key]) ? out[key].concat(next) : [out[key], next];
} else {
out[key] = next;
}
i++;
}
} else {
@@ -63,7 +66,7 @@ Env (optional):
Commands:
whoami
send --to <email> --subject <text> --body <text> [--html]
send --to <email> --subject <text> --body <text> [--html] [--attach <file>]
search-mail --query <gmail query> [--max 10]
search-calendar --query <text> [--max 10] [--timeMin ISO] [--timeMax ISO] [--calendar primary]
create-event --summary <text> --start <ISO> --end <ISO> [--timeZone America/Chicago] [--description <text>] [--location <text>] [--calendar primary]
@@ -77,9 +80,46 @@ function assertRequired(opts, required) {
}
}
function makeRawEmail({ from, to, subject, body, isHtml = false }) {
function toArray(value) {
if (value == null || value === false) return [];
return Array.isArray(value) ? value : [value];
}
function getAttachmentContentType(filename) {
const ext = path.extname(filename).toLowerCase();
if (ext === '.pdf') return 'application/pdf';
if (ext === '.txt') return 'text/plain; charset="UTF-8"';
if (ext === '.html' || ext === '.htm') return 'text/html; charset="UTF-8"';
if (ext === '.json') return 'application/json';
return 'application/octet-stream';
}
function wrapBase64(base64) {
return base64.match(/.{1,76}/g)?.join('\r\n') || '';
}
function loadAttachments(attachArg) {
return toArray(attachArg).map((filePath) => {
const absolutePath = path.resolve(filePath);
if (!fs.existsSync(absolutePath)) {
throw new Error(`Attachment file not found: ${absolutePath}`);
}
return {
filename: path.basename(absolutePath),
contentType: getAttachmentContentType(absolutePath),
data: fs.readFileSync(absolutePath).toString('base64'),
};
});
}
function makeRawEmail({ from, to, subject, body, isHtml = false, attachments = [] }) {
const contentType = isHtml ? 'text/html; charset="UTF-8"' : 'text/plain; charset="UTF-8"';
const msg = [
const normalizedAttachments = attachments.filter(Boolean);
let msg;
if (normalizedAttachments.length === 0) {
msg = [
`From: ${from}`,
`To: ${to}`,
`Subject: ${subject}`,
@@ -88,6 +128,34 @@ function makeRawEmail({ from, to, subject, body, isHtml = false }) {
'',
body,
].join('\r\n');
} else {
const boundary = `gw-boundary-${Date.now().toString(36)}-${Math.random().toString(36).slice(2, 10)}`;
msg = [
`From: ${from}`,
`To: ${to}`,
`Subject: ${subject}`,
'MIME-Version: 1.0',
`Content-Type: multipart/mixed; boundary="${boundary}"`,
'',
`--${boundary}`,
`Content-Type: ${contentType}`,
'Content-Transfer-Encoding: 7bit',
'',
body,
'',
...normalizedAttachments.flatMap((attachment) => [
`--${boundary}`,
`Content-Type: ${attachment.contentType}; name="${attachment.filename}"`,
'Content-Transfer-Encoding: base64',
`Content-Disposition: attachment; filename="${attachment.filename}"`,
'',
wrapBase64(attachment.data),
'',
]),
`--${boundary}--`,
'',
].join('\r\n');
}
return Buffer.from(msg)
.toString('base64')
@@ -97,6 +165,7 @@ function makeRawEmail({ from, to, subject, body, isHtml = false }) {
}
async function getClients() {
const { google } = require('googleapis');
const keyPath = resolveKeyPath();
if (!keyPath) {
throw new Error('Service account key not found. Set GW_KEY_PATH or place the file in ~/.openclaw/workspace/.clawdbot/credentials/google-workspace/service-account.json');
@@ -132,6 +201,7 @@ async function cmdSend(clients, opts) {
subject: opts.subject,
body: opts.body,
isHtml: !!opts.html,
attachments: loadAttachments(opts.attach),
});
const res = await clients.gmail.users.messages.send({
@@ -222,7 +292,7 @@ async function cmdCreateEvent(clients, opts) {
console.log(JSON.stringify({ ok: true, id: res.data.id, htmlLink: res.data.htmlLink }, null, 2));
}
(async function main() {
async function main() {
try {
const args = parseArgs(process.argv.slice(2));
const cmd = args._[0];
@@ -245,4 +315,19 @@ async function cmdCreateEvent(clients, opts) {
console.error(`ERROR: ${err.message}`);
process.exit(1);
}
})();
}
if (require.main === module) {
main();
}
module.exports = {
getAttachmentContentType,
loadAttachments,
main,
makeRawEmail,
parseArgs,
resolveKeyPath,
toArray,
wrapBase64,
};
+34
View File
@@ -0,0 +1,34 @@
const test = require('node:test');
const assert = require('node:assert/strict');
const { spawnSync } = require('node:child_process');
const path = require('node:path');
const { makeRawEmail } = require('./gw.js');
test('help documents attachment support for send', () => {
const gwPath = path.join(__dirname, 'gw.js');
const result = spawnSync(process.execPath, [gwPath, 'help'], { encoding: 'utf8' });
assert.equal(result.status, 0, result.stderr);
assert.match(result.stdout, /send --to <email> --subject <text> --body <text> \[--html\] \[--attach <file>\]/);
});
test('makeRawEmail builds multipart messages when attachments are present', () => {
const raw = makeRawEmail({
from: 'stefano@fiorinis.com',
to: 'stefano@fiorinis.com',
subject: 'Attachment test',
body: 'Attached PDF.',
attachments: [
{
filename: 'report.pdf',
contentType: 'application/pdf',
data: Buffer.from('%PDF-1.4\n%test\n').toString('base64'),
},
],
});
const decoded = Buffer.from(raw, 'base64').toString('utf8');
assert.match(decoded, /Content-Type: multipart\/mixed; boundary=/);
assert.match(decoded, /Content-Disposition: attachment; filename="report\.pdf"/);
assert.match(decoded, /Content-Type: application\/pdf; name="report\.pdf"/);
});
+1 -1
View File
@@ -4,7 +4,7 @@
"description": "",
"main": "gw.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
"test": "node --test"
},
"keywords": [],
"author": "",
+7
View File
@@ -0,0 +1,7 @@
node_modules/
dist/
coverage/
*.log
tmp/
out/
*.real.html
+42
View File
@@ -0,0 +1,42 @@
---
name: amazon-shopping
description: Search amazon.com shopping results with product filters using the local web-automation skill. Use when the user asks to find, compare, filter, or summarize Amazon products by description, price, delivery, specs, review count, star rating, or star distribution.
---
# Amazon Shopping
Use this skill for read-only Amazon product discovery and comparison.
## First Checks
Verify the browser dependency before live use:
```bash
openclaw skills info web-automation
node "$HOME/.openclaw/workspace/skills/web-automation/scripts/check-install.js"
```
## Search Products
Run the helper from the installed skill directory:
```bash
"$HOME/.openclaw/workspace/skills/amazon-shopping/scripts/search-products" '<user request>' --json --markdown
```
Use single quotes around product requests that contain dollar amounts so the shell does not expand `$4` or similar text. Use `--limit N`; `--max N` is accepted as a compatibility alias. If your execution tool supports a timeout option, set it to at least 600 seconds for live runs with detail enrichment. Use `--skip-details` only for a quick preview or when the user does not need specifications and delivery details.
Default to at most 15 products unless the user asks for a different count. For requested counts above 30, ask before continuing or split the request into batches. Always include source URLs, report missing fields explicitly, and do not claim review histogram data unless it was visible and extracted.
For user-facing answers, use the generated chat-safe result blocks as the presentation template. Keep the direct `Link: https://...` line for every product because WhatsApp and Telegram do not reliably render markdown tables. Do not rewrite Prime or delivery status as verified unless the helper marks it verified.
Supported filters include minimum rating, minimum reviews, maximum price, maximum unit price, minimum width in inches, Prime delivery, delivery by today/tomorrow/overnight, and sort by price. Natural language such as `77 inches or wider`, `shipped with Prime`, `delivery by tomorrow`, `overnight shipping`, and `top 10 by price` is parsed automatically. CLI flags are also available: `--min-width`, `--require-prime`, `--delivery-by`, and `--sort-by price`.
## Guardrails
This skill is for operator-directed, read-only product research. Before live scraping, the helper checks Amazon robots directives for planned paths. Do not automate sign-in, cart, purchase, wishlist, review submission, review-page crawling, CAPTCHA bypass, or blocked-page bypass. If Amazon returns a challenge or block page, stop and report that status.
Read references when needed:
- `references/amazon-data-map.md` for fields and selectors.
- `references/web-automation-prompts.md` for browser extraction prompts.
- `references/compliance-and-failure-modes.md` for blocked-page and unknown-field behavior.
+743
View File
@@ -0,0 +1,743 @@
{
"name": "amazon-shopping-scripts",
"version": "1.0.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "amazon-shopping-scripts",
"version": "1.0.0",
"dependencies": {
"minimist": "^1.2.8",
"node-html-parser": "^7.1.0"
},
"devDependencies": {
"@types/minimist": "^1.2.5",
"@types/node": "^24.8.1",
"tsx": "^4.20.6",
"typescript": "^5.9.3"
}
},
"node_modules/@esbuild/aix-ppc64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.27.7.tgz",
"integrity": "sha512-EKX3Qwmhz1eMdEJokhALr0YiD0lhQNwDqkPYyPhiSwKrh7/4KRjQc04sZ8db+5DVVnZ1LmbNDI1uAMPEUBnQPg==",
"cpu": [
"ppc64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"aix"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-arm": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.27.7.tgz",
"integrity": "sha512-jbPXvB4Yj2yBV7HUfE2KHe4GJX51QplCN1pGbYjvsyCZbQmies29EoJbkEc+vYuU5o45AfQn37vZlyXy4YJ8RQ==",
"cpu": [
"arm"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-arm64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.27.7.tgz",
"integrity": "sha512-62dPZHpIXzvChfvfLJow3q5dDtiNMkwiRzPylSCfriLvZeq0a1bWChrGx/BbUbPwOrsWKMn8idSllklzBy+dgQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-x64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.27.7.tgz",
"integrity": "sha512-x5VpMODneVDb70PYV2VQOmIUUiBtY3D3mPBG8NxVk5CogneYhkR7MmM3yR/uMdITLrC1ml/NV1rj4bMJuy9MCg==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/darwin-arm64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.27.7.tgz",
"integrity": "sha512-5lckdqeuBPlKUwvoCXIgI2D9/ABmPq3Rdp7IfL70393YgaASt7tbju3Ac+ePVi3KDH6N2RqePfHnXkaDtY9fkw==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/darwin-x64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.27.7.tgz",
"integrity": "sha512-rYnXrKcXuT7Z+WL5K980jVFdvVKhCHhUwid+dDYQpH+qu+TefcomiMAJpIiC2EM3Rjtq0sO3StMV/+3w3MyyqQ==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/freebsd-arm64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.27.7.tgz",
"integrity": "sha512-B48PqeCsEgOtzME2GbNM2roU29AMTuOIN91dsMO30t+Ydis3z/3Ngoj5hhnsOSSwNzS+6JppqWsuhTp6E82l2w==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/freebsd-x64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.27.7.tgz",
"integrity": "sha512-jOBDK5XEjA4m5IJK3bpAQF9/Lelu/Z9ZcdhTRLf4cajlB+8VEhFFRjWgfy3M1O4rO2GQ/b2dLwCUGpiF/eATNQ==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-arm": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.27.7.tgz",
"integrity": "sha512-RkT/YXYBTSULo3+af8Ib0ykH8u2MBh57o7q/DAs3lTJlyVQkgQvlrPTnjIzzRPQyavxtPtfg0EopvDyIt0j1rA==",
"cpu": [
"arm"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-arm64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.27.7.tgz",
"integrity": "sha512-RZPHBoxXuNnPQO9rvjh5jdkRmVizktkT7TCDkDmQ0W2SwHInKCAV95GRuvdSvA7w4VMwfCjUiPwDi0ZO6Nfe9A==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-ia32": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.27.7.tgz",
"integrity": "sha512-GA48aKNkyQDbd3KtkplYWT102C5sn/EZTY4XROkxONgruHPU72l+gW+FfF8tf2cFjeHaRbWpOYa/uRBz/Xq1Pg==",
"cpu": [
"ia32"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-loong64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.27.7.tgz",
"integrity": "sha512-a4POruNM2oWsD4WKvBSEKGIiWQF8fZOAsycHOt6JBpZ+JN2n2JH9WAv56SOyu9X5IqAjqSIPTaJkqN8F7XOQ5Q==",
"cpu": [
"loong64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-mips64el": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.27.7.tgz",
"integrity": "sha512-KabT5I6StirGfIz0FMgl1I+R1H73Gp0ofL9A3nG3i/cYFJzKHhouBV5VWK1CSgKvVaG4q1RNpCTR2LuTVB3fIw==",
"cpu": [
"mips64el"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-ppc64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.27.7.tgz",
"integrity": "sha512-gRsL4x6wsGHGRqhtI+ifpN/vpOFTQtnbsupUF5R5YTAg+y/lKelYR1hXbnBdzDjGbMYjVJLJTd2OFmMewAgwlQ==",
"cpu": [
"ppc64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-riscv64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.27.7.tgz",
"integrity": "sha512-hL25LbxO1QOngGzu2U5xeXtxXcW+/GvMN3ejANqXkxZ/opySAZMrc+9LY/WyjAan41unrR3YrmtTsUpwT66InQ==",
"cpu": [
"riscv64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-s390x": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.27.7.tgz",
"integrity": "sha512-2k8go8Ycu1Kb46vEelhu1vqEP+UeRVj2zY1pSuPdgvbd5ykAw82Lrro28vXUrRmzEsUV0NzCf54yARIK8r0fdw==",
"cpu": [
"s390x"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-x64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.27.7.tgz",
"integrity": "sha512-hzznmADPt+OmsYzw1EE33ccA+HPdIqiCRq7cQeL1Jlq2gb1+OyWBkMCrYGBJ+sxVzve2ZJEVeePbLM2iEIZSxA==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/netbsd-arm64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.27.7.tgz",
"integrity": "sha512-b6pqtrQdigZBwZxAn1UpazEisvwaIDvdbMbmrly7cDTMFnw/+3lVxxCTGOrkPVnsYIosJJXAsILG9XcQS+Yu6w==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/netbsd-x64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.27.7.tgz",
"integrity": "sha512-OfatkLojr6U+WN5EDYuoQhtM+1xco+/6FSzJJnuWiUw5eVcicbyK3dq5EeV/QHT1uy6GoDhGbFpprUiHUYggrw==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openbsd-arm64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.27.7.tgz",
"integrity": "sha512-AFuojMQTxAz75Fo8idVcqoQWEHIXFRbOc1TrVcFSgCZtQfSdc1RXgB3tjOn/krRHENUB4j00bfGjyl2mJrU37A==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openbsd-x64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.27.7.tgz",
"integrity": "sha512-+A1NJmfM8WNDv5CLVQYJ5PshuRm/4cI6WMZRg1by1GwPIQPCTs1GLEUHwiiQGT5zDdyLiRM/l1G0Pv54gvtKIg==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openharmony-arm64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.27.7.tgz",
"integrity": "sha512-+KrvYb/C8zA9CU/g0sR6w2RBw7IGc5J2BPnc3dYc5VJxHCSF1yNMxTV5LQ7GuKteQXZtspjFbiuW5/dOj7H4Yw==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"openharmony"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/sunos-x64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.27.7.tgz",
"integrity": "sha512-ikktIhFBzQNt/QDyOL580ti9+5mL/YZeUPKU2ivGtGjdTYoqz6jObj6nOMfhASpS4GU4Q/Clh1QtxWAvcYKamA==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"sunos"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-arm64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.27.7.tgz",
"integrity": "sha512-7yRhbHvPqSpRUV7Q20VuDwbjW5kIMwTHpptuUzV+AA46kiPze5Z7qgt6CLCK3pWFrHeNfDd1VKgyP4O+ng17CA==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-ia32": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.27.7.tgz",
"integrity": "sha512-SmwKXe6VHIyZYbBLJrhOoCJRB/Z1tckzmgTLfFYOfpMAx63BJEaL9ExI8x7v0oAO3Zh6D/Oi1gVxEYr5oUCFhw==",
"cpu": [
"ia32"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-x64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.27.7.tgz",
"integrity": "sha512-56hiAJPhwQ1R4i+21FVF7V8kSD5zZTdHcVuRFMW0hn753vVfQN8xlx4uOPT4xoGH0Z/oVATuR82AiqSTDIpaHg==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@types/minimist": {
"version": "1.2.5",
"resolved": "https://registry.npmjs.org/@types/minimist/-/minimist-1.2.5.tgz",
"integrity": "sha512-hov8bUuiLiyFPGyFPE1lwWhmzYbirOXQNNo40+y3zow8aFVTeyn3VWL0VFFfdNddA8S4Vf0Tc062rzyNr7Paag==",
"dev": true,
"license": "MIT"
},
"node_modules/@types/node": {
"version": "24.12.2",
"resolved": "https://registry.npmjs.org/@types/node/-/node-24.12.2.tgz",
"integrity": "sha512-A1sre26ke7HDIuY/M23nd9gfB+nrmhtYyMINbjI1zHJxYteKR6qSMX56FsmjMcDb3SMcjJg5BiRRgOCC/yBD0g==",
"dev": true,
"license": "MIT",
"dependencies": {
"undici-types": "~7.16.0"
}
},
"node_modules/boolbase": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/boolbase/-/boolbase-1.0.0.tgz",
"integrity": "sha512-JZOSA7Mo9sNGB8+UjSgzdLtokWAky1zbztM3WRLCbZ70/3cTANmQmOdR7y2g+J0e2WXywy1yS468tY+IruqEww==",
"license": "ISC"
},
"node_modules/css-select": {
"version": "5.2.2",
"resolved": "https://registry.npmjs.org/css-select/-/css-select-5.2.2.tgz",
"integrity": "sha512-TizTzUddG/xYLA3NXodFM0fSbNizXjOKhqiQQwvhlspadZokn1KDy0NZFS0wuEubIYAV5/c1/lAr0TaaFXEXzw==",
"license": "BSD-2-Clause",
"dependencies": {
"boolbase": "^1.0.0",
"css-what": "^6.1.0",
"domhandler": "^5.0.2",
"domutils": "^3.0.1",
"nth-check": "^2.0.1"
},
"funding": {
"url": "https://github.com/sponsors/fb55"
}
},
"node_modules/css-what": {
"version": "6.2.2",
"resolved": "https://registry.npmjs.org/css-what/-/css-what-6.2.2.tgz",
"integrity": "sha512-u/O3vwbptzhMs3L1fQE82ZSLHQQfto5gyZzwteVIEyeaY5Fc7R4dapF/BvRoSYFeqfBk4m0V1Vafq5Pjv25wvA==",
"license": "BSD-2-Clause",
"engines": {
"node": ">= 6"
},
"funding": {
"url": "https://github.com/sponsors/fb55"
}
},
"node_modules/dom-serializer": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/dom-serializer/-/dom-serializer-2.0.0.tgz",
"integrity": "sha512-wIkAryiqt/nV5EQKqQpo3SToSOV9J0DnbJqwK7Wv/Trc92zIAYZ4FlMu+JPFW1DfGFt81ZTCGgDEabffXeLyJg==",
"license": "MIT",
"dependencies": {
"domelementtype": "^2.3.0",
"domhandler": "^5.0.2",
"entities": "^4.2.0"
},
"funding": {
"url": "https://github.com/cheeriojs/dom-serializer?sponsor=1"
}
},
"node_modules/domelementtype": {
"version": "2.3.0",
"resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-2.3.0.tgz",
"integrity": "sha512-OLETBj6w0OsagBwdXnPdN0cnMfF9opN69co+7ZrbfPGrdpPVNBUj02spi6B1N7wChLQiPn4CSH/zJvXw56gmHw==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/fb55"
}
],
"license": "BSD-2-Clause"
},
"node_modules/domhandler": {
"version": "5.0.3",
"resolved": "https://registry.npmjs.org/domhandler/-/domhandler-5.0.3.tgz",
"integrity": "sha512-cgwlv/1iFQiFnU96XXgROh8xTeetsnJiDsTc7TYCLFd9+/WNkIqPTxiM/8pSd8VIrhXGTf1Ny1q1hquVqDJB5w==",
"license": "BSD-2-Clause",
"dependencies": {
"domelementtype": "^2.3.0"
},
"engines": {
"node": ">= 4"
},
"funding": {
"url": "https://github.com/fb55/domhandler?sponsor=1"
}
},
"node_modules/domutils": {
"version": "3.2.2",
"resolved": "https://registry.npmjs.org/domutils/-/domutils-3.2.2.tgz",
"integrity": "sha512-6kZKyUajlDuqlHKVX1w7gyslj9MPIXzIFiz/rGu35uC1wMi+kMhQwGhl4lt9unC9Vb9INnY9Z3/ZA3+FhASLaw==",
"license": "BSD-2-Clause",
"dependencies": {
"dom-serializer": "^2.0.0",
"domelementtype": "^2.3.0",
"domhandler": "^5.0.3"
},
"funding": {
"url": "https://github.com/fb55/domutils?sponsor=1"
}
},
"node_modules/entities": {
"version": "4.5.0",
"resolved": "https://registry.npmjs.org/entities/-/entities-4.5.0.tgz",
"integrity": "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw==",
"license": "BSD-2-Clause",
"engines": {
"node": ">=0.12"
},
"funding": {
"url": "https://github.com/fb55/entities?sponsor=1"
}
},
"node_modules/esbuild": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.27.7.tgz",
"integrity": "sha512-IxpibTjyVnmrIQo5aqNpCgoACA/dTKLTlhMHihVHhdkxKyPO1uBBthumT0rdHmcsk9uMonIWS0m4FljWzILh3w==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
"bin": {
"esbuild": "bin/esbuild"
},
"engines": {
"node": ">=18"
},
"optionalDependencies": {
"@esbuild/aix-ppc64": "0.27.7",
"@esbuild/android-arm": "0.27.7",
"@esbuild/android-arm64": "0.27.7",
"@esbuild/android-x64": "0.27.7",
"@esbuild/darwin-arm64": "0.27.7",
"@esbuild/darwin-x64": "0.27.7",
"@esbuild/freebsd-arm64": "0.27.7",
"@esbuild/freebsd-x64": "0.27.7",
"@esbuild/linux-arm": "0.27.7",
"@esbuild/linux-arm64": "0.27.7",
"@esbuild/linux-ia32": "0.27.7",
"@esbuild/linux-loong64": "0.27.7",
"@esbuild/linux-mips64el": "0.27.7",
"@esbuild/linux-ppc64": "0.27.7",
"@esbuild/linux-riscv64": "0.27.7",
"@esbuild/linux-s390x": "0.27.7",
"@esbuild/linux-x64": "0.27.7",
"@esbuild/netbsd-arm64": "0.27.7",
"@esbuild/netbsd-x64": "0.27.7",
"@esbuild/openbsd-arm64": "0.27.7",
"@esbuild/openbsd-x64": "0.27.7",
"@esbuild/openharmony-arm64": "0.27.7",
"@esbuild/sunos-x64": "0.27.7",
"@esbuild/win32-arm64": "0.27.7",
"@esbuild/win32-ia32": "0.27.7",
"@esbuild/win32-x64": "0.27.7"
}
},
"node_modules/fsevents": {
"version": "2.3.3",
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz",
"integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
}
},
"node_modules/get-tsconfig": {
"version": "4.14.0",
"resolved": "https://registry.npmjs.org/get-tsconfig/-/get-tsconfig-4.14.0.tgz",
"integrity": "sha512-yTb+8DXzDREzgvYmh6s9vHsSVCHeC0G3PI5bEXNBHtmshPnO+S5O7qgLEOn0I5QvMy6kpZN8K1NKGyilLb93wA==",
"dev": true,
"license": "MIT",
"dependencies": {
"resolve-pkg-maps": "^1.0.0"
},
"funding": {
"url": "https://github.com/privatenumber/get-tsconfig?sponsor=1"
}
},
"node_modules/he": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/he/-/he-1.2.0.tgz",
"integrity": "sha512-F/1DnUGPopORZi0ni+CvrCgHQ5FyEAHRLSApuYWMmrbSwoN2Mn/7k+Gl38gJnR7yyDZk6WLXwiGod1JOWNDKGw==",
"license": "MIT",
"bin": {
"he": "bin/he"
}
},
"node_modules/minimist": {
"version": "1.2.8",
"resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz",
"integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==",
"license": "MIT",
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/node-html-parser": {
"version": "7.1.0",
"resolved": "https://registry.npmjs.org/node-html-parser/-/node-html-parser-7.1.0.tgz",
"integrity": "sha512-iJo8b2uYGT40Y8BTyy5ufL6IVbN8rbm/1QK2xffXU/1a/v3AAa0d1YAoqBNYqaS4R/HajkWIpIfdE6KcyFh1AQ==",
"license": "MIT",
"dependencies": {
"css-select": "^5.1.0",
"he": "1.2.0"
}
},
"node_modules/nth-check": {
"version": "2.1.1",
"resolved": "https://registry.npmjs.org/nth-check/-/nth-check-2.1.1.tgz",
"integrity": "sha512-lqjrjmaOoAnWfMmBPL+XNnynZh2+swxiX3WUE0s4yEHI6m+AwrK2UZOimIRl3X/4QctVqS8AiZjFqyOGrMXb/w==",
"license": "BSD-2-Clause",
"dependencies": {
"boolbase": "^1.0.0"
},
"funding": {
"url": "https://github.com/fb55/nth-check?sponsor=1"
}
},
"node_modules/resolve-pkg-maps": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/resolve-pkg-maps/-/resolve-pkg-maps-1.0.0.tgz",
"integrity": "sha512-seS2Tj26TBVOC2NIc2rOe2y2ZO7efxITtLZcGSOnHHNOQ7CkiUBfw0Iw2ck6xkIhPwLhKNLS8BO+hEpngQlqzw==",
"dev": true,
"license": "MIT",
"funding": {
"url": "https://github.com/privatenumber/resolve-pkg-maps?sponsor=1"
}
},
"node_modules/tsx": {
"version": "4.21.0",
"resolved": "https://registry.npmjs.org/tsx/-/tsx-4.21.0.tgz",
"integrity": "sha512-5C1sg4USs1lfG0GFb2RLXsdpXqBSEhAaA/0kPL01wxzpMqLILNxIxIOKiILz+cdg/pLnOUxFYOR5yhHU666wbw==",
"dev": true,
"license": "MIT",
"dependencies": {
"esbuild": "~0.27.0",
"get-tsconfig": "^4.7.5"
},
"bin": {
"tsx": "dist/cli.mjs"
},
"engines": {
"node": ">=18.0.0"
},
"optionalDependencies": {
"fsevents": "~2.3.3"
}
},
"node_modules/typescript": {
"version": "5.9.3",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz",
"integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"tsc": "bin/tsc",
"tsserver": "bin/tsserver"
},
"engines": {
"node": ">=14.17"
}
},
"node_modules/undici-types": {
"version": "7.16.0",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-7.16.0.tgz",
"integrity": "sha512-Zz+aZWSj8LE6zoxD+xrjh4VfkIG8Ya6LvYkZqtUQGJPZjYl53ypCaUwWqo7eI0x66KBGeRo+mlBEkMSeSZ38Nw==",
"dev": true,
"license": "MIT"
}
}
}
+22
View File
@@ -0,0 +1,22 @@
{
"name": "amazon-shopping-scripts",
"version": "1.0.0",
"description": "Amazon shopping helper CLI for OpenClaw skills",
"type": "module",
"scripts": {
"amazon-shopping": "tsx src/cli.ts",
"lint": "node scripts/lint.mjs",
"test": "node --import tsx --test tests/*.test.ts",
"typecheck": "tsc --noEmit"
},
"dependencies": {
"minimist": "^1.2.8",
"node-html-parser": "^7.1.0"
},
"devDependencies": {
"@types/minimist": "^1.2.5",
"@types/node": "^24.8.1",
"tsx": "^4.20.6",
"typescript": "^5.9.3"
}
}
@@ -0,0 +1,52 @@
# Amazon Data Map
Use this reference when deciding which visible Amazon fields can be reported by `amazon-shopping`.
## Product Search Fields
Search result cards should be treated as candidates, not final truth. Prefer cards with a non-empty `data-asin` value. Extract only visible data from the rendered search page:
| Output field | Search-page source | Notes |
|---|---|---|
| `asin` | `data-asin` on result card | Required for normalized detail links. |
| `title` | product heading or product link text | Trim sponsored/accessibility boilerplate. |
| `url` | product link | Normalize to `https://www.amazon.com/dp/<ASIN>` when safe. |
| `imageUrl` | visible product image `src` | Optional. |
| `price` | visible `.a-price` text | Do not infer absent prices from snippets. |
| `rating` | `aria-label` or visible text like `4.6 out of 5 stars` | Optional. |
| `reviewCount` | text near rating like `1,234 ratings` | Optional. |
| `delivery.display` | visible delivery promise text | Optional and ZIP/session dependent. |
| `delivery.prime` | visible Prime badge, Prime icon class, `aria-label`, `alt`, or delivery text | Optional and ZIP/session dependent. Preserve a true search-card Prime signal when detail text omits the literal word Prime. |
| `isSponsored` | visible sponsored marker | Sponsored results may be included but must be labeled. |
## Detail Page Fields
Open only normalized product detail URLs under `/dp/<ASIN>` or `/gp/product/<ASIN>`. Extract visible fields:
| Output field | Detail-page source | Notes |
|---|---|---|
| `title` | `#productTitle` or equivalent heading | Detail title can replace search title. |
| `price` | buy-box/current price selectors | Variant pages can omit price. |
| `delivery` | delivery message near buy box | Report as text, not guaranteed. |
| `availability` | availability block | Optional. |
| `seller` | seller/ships-from visible text | Optional. |
| `bullets` | feature bullets list | Trim empty and hidden items. |
| `specs` | product overview/details/technical tables | Preserve name/value pairs. |
| `starBreakdown` | visible customer-review histogram | Percent or count basis only. Do not crawl review pages. |
## Filter Semantics
- `over 200 reviews` means `reviewCount > 200`.
- `at least 200 reviews` means `reviewCount >= 200`.
- `more than 4.5 stars` means `rating > 4.5`.
- `4.5 stars or better` means `rating >= 4.5`.
- `less than $4 each` means visible unit price first, then high-confidence unit-count inference. Unknown unit prices do not pass strict unit-price filters.
- `77 inches or wider` means the overall product width must be `>= 77` inches. Prefer product/item dimensions with an explicit `W` component; ignore seat, arm, door, package, and cushion widths.
- `shipped with Prime` / `Prime shipping` means a visible Prime signal must be detected on the search card or detail page.
- `delivery by tomorrow` and `overnight shipping` require visible delivery text that indicates tomorrow, overnight, next-day, or one-day delivery.
- `top 10 by price` sorts passing products by displayed product price ascending.
- Missing fields must be represented as `null` or noted in `missingFields` / `extractionNotes`; never fabricate values.
## Official Alternatives
Amazon Business Product Search API and Product Advertising API are official API paths for structured product data when the operator has credentials. This skill uses bounded web automation because the current install request requires `web-automation` scraping.
@@ -0,0 +1,39 @@
# Compliance And Failure Modes
This reference is operational guidance, not legal advice. The operator is responsible for making sure a run complies with Amazon terms, robots directives, local law, and account obligations.
## Required Guardrails
- Fetch and evaluate `https://www.amazon.com/robots.txt` before live scraping planned Amazon paths.
- Stop if the effective rules disallow the planned search or detail paths.
- Do not automate sign-in, checkout, cart, wishlist, review submission, customer-review pages, reviewer profiles, or any disallowed path.
- Do not bypass CAPTCHA, bot checks, blocked pages, or access-denied pages.
- Do not print cookies, profile state, session storage, or account/location-specific browser data.
## Allowed Scope
Allowed behavior is bounded read-only product research over search result pages and normalized product detail pages:
- `/s?k=<query>` search results.
- `/dp/<ASIN>` product details.
- `/gp/product/<ASIN>` product details.
Review data is limited to visible summary ratings/counts and visible histogram rows on search/detail pages. Do not navigate to `/product-reviews`, `/review`, `/gp/customer-reviews`, or review AJAX endpoints.
## Failure Modes
Return a structured warning and do not claim success when any of these happen:
- CAPTCHA or bot-check page.
- Sign-in wall.
- HTTP 429 or 503 that remains after the bounded retry budget.
- Robots rules disallow a planned path.
- Product markup changes enough that required fields cannot be found.
- Amazon returns localized, personalized, or ZIP/session-dependent delivery text that cannot be verified.
## Output Rules
- Unknown fields stay unknown.
- Partial extraction is acceptable only when the response includes warnings and missing-field notes.
- Sponsored products can be returned by default but must be labeled.
- Counts above 30 require operator confirmation or batch splitting.
@@ -0,0 +1,27 @@
# Web-Automation Prompts
Use these patterns when debugging or extending the `amazon-shopping` browser workflow. The TypeScript helper is the default interface; these prompts document the intended rendered-page behavior.
## Search Page
```text
Use the installed web-automation skill. Open https://www.amazon.com/s?k=<encoded query>. Wait for rendered search results. If a CAPTCHA, bot check, sign-in wall, or access denied page appears, stop and return a challenge status. Otherwise extract visible product cards with ASIN, title, product URL, displayed price, rating text, review count text, delivery text, Prime badge/icon/aria/alt signal, sponsor marker, and image URL. Do not open cart, sign-in, wishlist, or review-listing pages.
```
## Detail Page
```text
For each candidate product detail URL under /dp/<ASIN> or /gp/product/<ASIN>, open the page slowly one at a time. Extract title, canonical link, visible buy-box/current price, delivery summary, Prime badge/icon/aria/alt signal, availability, seller when visible, feature bullets, product details/specification table rows, rating score, review count, and visible customer-review histogram percentages if present on the detail page. Do not navigate to /product-reviews or reviewer profile pages.
```
## Pagination
```text
Follow only the visible Amazon pagination control for the next search page, or construct page=<n> only after the current page exposes normal search results and no challenge/block. Stop when enough candidates have been collected, no next page exists, a challenge appears, or maxSearchPages is reached.
```
## Robustness Notes
- Prefer Playwright locator/actionability behavior and bounded waits over fixed sleeps.
- Never follow sponsored redirect URLs, sign-in links, cart links, wishlist links, or review-page links.
- Return partial results with warnings when Amazon markup changes or fields are hidden.
+54
View File
@@ -0,0 +1,54 @@
#!/usr/bin/env node
import { readdir, readFile, stat } from "node:fs/promises";
import { join, relative } from "node:path";
const root = new URL("..", import.meta.url).pathname;
const scannedExtensions = new Set([".md", ".json", ".ts", ".js", ".mjs", ".sh"]);
const installSpecificPath = ["", "Users", "stefano"].join("/");
const forbidden = [
{
pattern: installSpecificPath,
message: "Source files must not hardcode this install path"
}
];
function extensionOf(path) {
const dot = path.lastIndexOf(".");
return dot === -1 ? "" : path.slice(dot);
}
async function walk(dir) {
const entries = await readdir(dir);
const files = [];
for (const entry of entries) {
if (["node_modules", "dist", "coverage", "tmp", "out"].includes(entry)) {
continue;
}
const path = join(dir, entry);
const info = await stat(path);
if (info.isDirectory()) {
files.push(...await walk(path));
} else if (scannedExtensions.has(extensionOf(path)) || entry === "SKILL.md") {
files.push(path);
}
}
return files;
}
const failures = [];
for (const file of await walk(root)) {
const text = await readFile(file, "utf8");
for (const rule of forbidden) {
if (text.includes(rule.pattern)) {
failures.push(`${relative(root, file)}: ${rule.message}`);
}
}
}
if (failures.length > 0) {
for (const failure of failures) {
console.error(failure);
}
process.exitCode = 1;
}
+13
View File
@@ -0,0 +1,13 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SKILL_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
TSX="${SKILL_DIR}/node_modules/.bin/tsx"
if [ ! -x "$TSX" ]; then
echo "Missing local dependencies. Run: cd \"$SKILL_DIR\" && npm install" >&2
exit 127
fi
exec "$TSX" "$SKILL_DIR/src/cli.ts" "$@"
+10
View File
@@ -0,0 +1,10 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SKILL_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
cd "$SKILL_DIR"
npm install
npm run lint
npm test
+228
View File
@@ -0,0 +1,228 @@
import { execFile } from "node:child_process";
import { pathToFileURL } from "node:url";
import { join } from "node:path";
import { promisify } from "node:util";
import { extractDetailPage } from "./detail-page.js";
import { applyFiltersAndLimit } from "./filters.js";
import { createResponse } from "./report.js";
import { extractSearchPage } from "./search-page.js";
import type { ProductSearchResult, SearchProductsRequest, SearchProductsResponse } from "./types.js";
import { resolveWebAutomationRuntime } from "./web-automation-runtime.js";
const execFileAsync = promisify(execFile);
const AMAZON_ROOT = "https://www.amazon.com";
const DEFAULT_WAIT_MS = 4500;
export type HttpClassification = "ok" | "retryable" | "challenge";
interface BrowserDeps {
fetchText?: (url: string) => Promise<string>;
sleep?: (ms: number) => Promise<void>;
now?: () => Date;
}
export function plannedAmazonPaths(asins: string[]): string[] {
return [
"/s",
...asins.flatMap((asin) => [`/dp/${asin}`, `/gp/product/${asin}`])
];
}
export function classifyHttpStatus(status: number | null | undefined): HttpClassification {
if (status === 429 || status === 503) return "retryable";
if (status === 401 || status === 403) return "challenge";
return "ok";
}
export function isPathAllowedByRobots(robots: string, userAgent: string, path: string): boolean {
const groups: Array<{ agents: string[]; disallows: string[] }> = [];
let current: { agents: string[]; disallows: string[] } | undefined;
let hasDirectives = false;
const lines = robots.split(/\r?\n/);
for (const rawLine of lines) {
const line = rawLine.replace(/#.*/, "").trim();
if (!line) continue;
const [rawKey, ...rest] = line.split(":");
const key = rawKey.trim().toLowerCase();
const value = rest.join(":").trim();
if (key === "user-agent") {
if (!current || hasDirectives) {
current = { agents: [], disallows: [] };
groups.push(current);
hasDirectives = false;
}
current.agents.push(value.toLowerCase());
continue;
}
if (key === "disallow") {
hasDirectives = true;
if (current && value) {
current.disallows.push(value);
}
}
}
const normalizedAgent = userAgent.toLowerCase();
const exactGroups = groups.filter((group) => group.agents.includes(normalizedAgent));
const matchedGroups = exactGroups.length > 0 ? exactGroups : groups.filter((group) => group.agents.includes("*"));
const disallows = matchedGroups.flatMap((group) => group.disallows);
return !disallows.some((rule) => path.startsWith(rule));
}
async function defaultFetchText(url: string): Promise<string> {
const response = await fetch(url);
return response.text();
}
async function checkRobots(paths: string[], deps: BrowserDeps): Promise<string[]> {
const warnings: string[] = [];
const robots = await (deps.fetchText ?? defaultFetchText)(`${AMAZON_ROOT}/robots.txt`);
for (const path of paths) {
if (!isPathAllowedByRobots(robots, "*", path)) {
warnings.push(`Amazon robots directives disallow planned path: ${path}`);
}
}
return warnings;
}
async function loadCloakBrowser(runtimeDir: string): Promise<{
ensureBinary?: () => Promise<void>;
launchContext: (options: Record<string, unknown>) => Promise<any>;
}> {
const moduleUrl = pathToFileURL(join(runtimeDir, "node_modules", "cloakbrowser", "dist", "index.js")).toString();
return import(moduleUrl) as Promise<any>;
}
async function checkRuntime(): Promise<string> {
const runtime = await resolveWebAutomationRuntime();
await execFileAsync(runtime.checkInstall.command, runtime.checkInstall.args, { cwd: runtime.checkInstall.cwd });
return runtime.scriptsDir;
}
function searchUrl(query: string, pageNumber: number): string {
const url = new URL("/s", AMAZON_ROOT);
url.searchParams.set("k", query);
if (pageNumber > 1) {
url.searchParams.set("page", String(pageNumber));
}
return url.toString();
}
async function pageHtml(page: any, url: string, deps: BrowserDeps): Promise<{ html: string; status: number | null }> {
let lastStatus: number | null = null;
for (let attempt = 0; attempt < 3; attempt += 1) {
const response = await page.goto(url, { waitUntil: "domcontentloaded", timeout: 45000 });
await page.waitForTimeout?.(DEFAULT_WAIT_MS);
lastStatus = response?.status?.() ?? null;
if (classifyHttpStatus(lastStatus) !== "retryable") {
return {
html: await page.content(),
status: lastStatus
};
}
await (deps.sleep ?? ((ms) => new Promise((resolve) => setTimeout(resolve, ms))))((2 ** attempt) * 1000 + Math.floor(Math.random() * 500));
}
return {
html: await page.content(),
status: lastStatus
};
}
async function enrichDetails(page: any, products: ProductSearchResult[], deps: BrowserDeps): Promise<ProductSearchResult[]> {
const enriched: ProductSearchResult[] = [];
for (const product of products) {
await (deps.sleep ?? ((ms) => new Promise((resolve) => setTimeout(resolve, ms))))(1500 + Math.floor(Math.random() * 1500));
const loaded = await pageHtml(page, product.url, deps);
const classification = classifyHttpStatus(loaded.status);
if (classification === "challenge") {
enriched.push({
...product,
extractionNotes: [...product.extractionNotes, "Detail page returned a challenge/block status."]
});
continue;
}
enriched.push(extractDetailPage(loaded.html, product));
}
return enriched;
}
export async function searchProducts(request: SearchProductsRequest, deps: BrowserDeps = {}): Promise<SearchProductsResponse> {
const warnings: string[] = [];
const robotsWarnings = await checkRobots(plannedAmazonPaths([]), deps);
if (robotsWarnings.length > 0) {
return createResponse({
query: request.query,
filters: request.filters,
limit: request.limit,
maxSearchPages: request.maxSearchPages,
results: [],
filteredOutCount: 0,
warnings: robotsWarnings,
now: deps.now
});
}
const runtimeDir = await checkRuntime();
const cloak = await loadCloakBrowser(runtimeDir);
await cloak.ensureBinary?.();
const context = await cloak.launchContext({
headless: process.env.CLOAKBROWSER_HEADLESS !== "false",
locale: "en-US",
viewport: { width: 1440, height: 900 },
humanize: true
});
const page = await context.newPage();
try {
const candidates: ProductSearchResult[] = [];
let nextUrl: string | undefined = searchUrl(request.query, 1);
for (let pageNumber = 1; pageNumber <= request.maxSearchPages && nextUrl; pageNumber += 1) {
const loaded = await pageHtml(page, nextUrl, deps);
const classification = classifyHttpStatus(loaded.status);
if (classification === "challenge" || classification === "retryable") {
warnings.push(`Amazon returned status ${loaded.status}; stopping without bypass.`);
break;
}
const extracted = extractSearchPage(loaded.html, nextUrl);
warnings.push(...extracted.warnings);
if (extracted.status === "challenge") {
break;
}
candidates.push(...extracted.products);
if (candidates.length >= request.limit * 3) {
break;
}
nextUrl = extracted.nextPageUrl ?? (pageNumber + 1 <= request.maxSearchPages ? searchUrl(request.query, pageNumber + 1) : undefined);
}
let detailCandidates = candidates;
if (!request.skipDetails) {
const detailPaths = plannedAmazonPaths(candidates.map((candidate) => candidate.asin)).filter((path) => path !== "/s");
const detailRobotsWarnings = await checkRobots(detailPaths, deps);
if (detailRobotsWarnings.length > 0) {
warnings.push(...detailRobotsWarnings, "Detail enrichment skipped because robots directives disallow at least one planned detail path.");
} else {
detailCandidates = await enrichDetails(page, candidates.slice(0, request.limit * 3), deps);
}
}
const filtered = applyFiltersAndLimit(detailCandidates, request.filters, request.limit);
return createResponse({
query: request.query,
filters: request.filters,
limit: request.limit,
maxSearchPages: request.maxSearchPages,
results: filtered.results,
filteredOutCount: filtered.filteredOutCount,
warnings,
now: deps.now
});
} finally {
await context.close();
}
}
+211
View File
@@ -0,0 +1,211 @@
#!/usr/bin/env node
import minimist from "minimist";
import { fileURLToPath } from "node:url";
import { searchProducts } from "./browser.js";
import { parseNaturalLanguageRequest } from "./query-parser.js";
import { createMarkdownReport } from "./report.js";
import type { ProductFilters, SearchProductsRequest, SearchProductsResponse } from "./types.js";
export interface CliDeps {
stdout: Pick<NodeJS.WriteStream, "write">;
stderr: Pick<NodeJS.WriteStream, "write">;
now?: () => Date;
searchProducts?: (request: SearchProductsRequest) => Promise<SearchProductsResponse>;
}
export function usage(): string {
return `amazon-shopping
Usage:
scripts/search-products "<product request>" [options]
scripts/search-products --query "<product request>" [options]
Options:
--json Print JSON output
--markdown Print markdown output
--limit N, --max N Maximum products to return (default: 15)
--allow-large-limit Permit limits above 30
--min-rating N Minimum rating score
--min-reviews N Minimum review count
--max-price N Maximum displayed product price
--max-unit-price N Maximum price per unit
--min-width N Minimum product width in inches
--require-prime Require Prime delivery verification
--delivery-by VALUE Require delivery timing, e.g. today, tomorrow, overnight
--max-search-pages N Search result pages to scan, 1-5 (default: 2)
--skip-details Do not open product detail pages
--dry-run Parse and print the planned request without Amazon network access
--help Show this help
`;
}
function parsePositiveInteger(value: unknown, name: string): number | undefined {
if (value === undefined) {
return undefined;
}
const parsed = Number(value);
if (!Number.isInteger(parsed) || parsed < 1) {
throw new Error(`${name} must be an integer greater than 0`);
}
return parsed;
}
function parseNumber(value: unknown, name: string): number | undefined {
if (value === undefined) {
return undefined;
}
const parsed = Number(value);
if (!Number.isFinite(parsed)) {
throw new Error(`${name} must be a number`);
}
return parsed;
}
export function buildSearchUrl(query: string): string {
return `https://www.amazon.com/s?k=${encodeURIComponent(query)}`;
}
export function parseCliRequest(argv: string[]): SearchProductsRequest {
const args = minimist(argv, {
boolean: ["help", "json", "markdown", "allow-large-limit", "dry-run", "skip-details", "require-prime"],
string: [
"query",
"limit",
"max",
"min-rating",
"min-reviews",
"max-price",
"max-unit-price",
"min-width",
"delivery-by",
"sort-by",
"max-search-pages"
],
alias: { h: "help", max: "limit" }
});
const rawQuery = String(args.query ?? args._.join(" ")).trim();
if (!rawQuery) {
throw new Error("A product query is required");
}
const natural = parseNaturalLanguageRequest(rawQuery);
const limit = parsePositiveInteger(args.limit, "limit") ?? natural.limit ?? 15;
if (limit > 30 && !args["allow-large-limit"]) {
throw new Error("Requested limits above 30 require --allow-large-limit or a batched run");
}
const maxSearchPages = parsePositiveInteger(args["max-search-pages"], "max-search-pages") ?? 2;
if (maxSearchPages > 5) {
throw new Error("max-search-pages must be 5 or less");
}
const filters: ProductFilters = { ...natural.filters };
const minRating = parseNumber(args["min-rating"], "min-rating");
const minReviews = parsePositiveInteger(args["min-reviews"], "min-reviews");
const maxPrice = parseNumber(args["max-price"], "max-price");
const maxUnitPrice = parseNumber(args["max-unit-price"], "max-unit-price");
const minWidth = parseNumber(args["min-width"], "min-width");
if (minRating !== undefined) filters.minRating = minRating;
if (minReviews !== undefined) filters.minReviews = minReviews;
if (maxPrice !== undefined) filters.maxPrice = maxPrice;
if (maxUnitPrice !== undefined) filters.maxUnitPrice = maxUnitPrice;
if (minWidth !== undefined) {
filters.minWidthInches = minWidth;
filters.widthComparison = "gte";
}
if (args["require-prime"]) filters.requirePrime = true;
if (args["delivery-by"]) filters.deliveryBy = String(args["delivery-by"]);
if (args["sort-by"]) {
const sortBy = String(args["sort-by"]);
if (sortBy !== "price" && sortBy !== "relevance") {
throw new Error("sort-by must be either price or relevance");
}
filters.sortBy = sortBy;
}
const json = Boolean(args.json);
const markdown = Boolean(args.markdown);
return {
query: natural.query || rawQuery,
filters,
limit,
maxSearchPages,
skipDetails: Boolean(args["skip-details"]),
dryRun: Boolean(args["dry-run"]),
output: json && markdown ? "both" : markdown ? "markdown" : "json"
};
}
function createDryRunResponse(request: SearchProductsRequest, now: () => Date): SearchProductsResponse {
return {
query: request.query,
filters: request.filters,
limit: request.limit,
maxSearchPages: request.maxSearchPages,
results: [],
filteredOutCount: 0,
warnings: [`Dry run only. Planned search URL: ${buildSearchUrl(request.query)}`],
source: {
site: "amazon.com",
scrapedAt: now().toISOString(),
automation: "web-automation/CloakBrowser"
}
};
}
async function defaultSearchProducts(request: SearchProductsRequest, deps: CliDeps): Promise<SearchProductsResponse> {
if (request.dryRun) {
return createDryRunResponse(request, deps.now ?? (() => new Date()));
}
return searchProducts(request, { now: deps.now });
}
function writeResponse(response: SearchProductsResponse, output: SearchProductsRequest["output"], deps: CliDeps): void {
if (output === "markdown") {
deps.stdout.write(createMarkdownReport(response));
return;
}
if (output === "both") {
deps.stdout.write(`${JSON.stringify(response, null, 2)}\n\n${createMarkdownReport(response)}`);
return;
}
deps.stdout.write(`${JSON.stringify(response, null, 2)}\n`);
}
export async function runCli(
argv: string[],
deps: CliDeps = { stdout: process.stdout, stderr: process.stderr }
): Promise<number> {
const rawArgs = minimist(argv, { boolean: ["help"], alias: { h: "help" } });
if (rawArgs.help || argv.length === 0) {
deps.stdout.write(usage());
return 0;
}
try {
const request = parseCliRequest(argv);
const response = deps.searchProducts
? await deps.searchProducts(request)
: await defaultSearchProducts(request, deps);
writeResponse(response, request.output, deps);
return 0;
} catch (error: unknown) {
const message = error instanceof Error ? error.message : String(error);
deps.stderr.write(`${message}\n`);
return 1;
}
}
if (process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]) {
runCli(process.argv.slice(2)).then((code) => {
process.exitCode = code;
}).catch((error: unknown) => {
const message = error instanceof Error ? error.message : String(error);
process.stderr.write(`${message}\n`);
process.exitCode = 1;
});
}
+160
View File
@@ -0,0 +1,160 @@
import { HTMLElement, parse } from "node-html-parser";
import { parseMoney, parseRating, parseReviewCount, parseStarBreakdown } from "./parsers.js";
import type { DeliverySummary, ProductSearchResult, ProductSpec } from "./types.js";
function textOf(node: HTMLElement | null | undefined): string {
return cleanText(node?.textContent ?? "");
}
function attrOf(node: HTMLElement | null | undefined, name: string): string {
return cleanText(node?.getAttribute(name) ?? "");
}
function cleanText(text: string): string {
return text
.replace(/\s+/g, " ")
.replace(/\s*\{".*$/g, "")
.trim();
}
function isScriptLike(text: string): boolean {
return /\(function\s*\(|window\.|P\.when|ue\.count|tracking\(\)|logShoppableMetrics|buying options|add to cart/i.test(text);
}
function firstText(root: HTMLElement, selectors: string[]): string {
for (const selector of selectors) {
const text = textOf(root.querySelector(selector));
if (text) {
return text;
}
}
return "";
}
function extractBullets(root: HTMLElement): string[] {
const spanBullets = root.querySelectorAll("#feature-bullets li span")
.map((node) => textOf(node))
.filter((text) => text && !/make sure this fits/i.test(text));
if (spanBullets.length > 0) {
return spanBullets;
}
return root.querySelectorAll("#feature-bullets li")
.map((node) => textOf(node))
.filter((text) => text && !/make sure this fits/i.test(text));
}
function extractSpecs(root: HTMLElement): ProductSpec[] {
const specs: ProductSpec[] = [];
const seen = new Set<string>();
const excludedNames = new Set(["customer reviews"]);
for (const row of root.querySelectorAll("tr")) {
const cells = row.querySelectorAll("th,td").map((cell) => textOf(cell)).filter(Boolean);
if (cells.length >= 2) {
const name = cells[0];
const value = cells.slice(1).join(" ");
const key = name.toLowerCase();
if (seen.has(key) || excludedNames.has(key) || isScriptLike(name) || isScriptLike(value)) {
continue;
}
seen.add(key);
specs.push({ name, value });
}
}
return specs;
}
function extractHistogramText(root: HTMLElement): string {
const rows = root.querySelectorAll("#histogramTable tr, [aria-label*='star'] tr");
const parts: string[] = [];
for (const row of rows) {
const cells = row.querySelectorAll("th,td").map((cell) => textOf(cell)).filter(Boolean);
if (cells.length >= 2) {
parts.push(`${cells[0]} ${cells[1]}`);
}
}
return parts.join(" ");
}
function deliveryFromText(text: string, primeSignal = false): DeliverySummary | undefined {
const display = text.replace(/\s+/g, " ").trim();
if (!display) {
return primeSignal ? { display: "Prime delivery available", prime: true } : undefined;
}
return {
display,
free: /\bfree\b/i.test(display),
prime: primeSignal || /\bprime\b/i.test(display)
};
}
function hasPrimeSignal(root: HTMLElement): boolean {
const attributeText = root.querySelectorAll("[id], [class], [aria-label], img[alt]")
.map((node) => [
attrOf(node, "id"),
attrOf(node, "class"),
attrOf(node, "aria-label"),
attrOf(node, "alt")
].join(" "))
.join(" ");
return /a-icon-prime|prime-logo|primeExclusive|primePopover|amazon\s+prime|\bprime\b/i.test(attributeText);
}
function mergeDelivery(detail: DeliverySummary | undefined, base: DeliverySummary | undefined): DeliverySummary | undefined {
if (!detail) {
return base;
}
if (!base) {
return detail;
}
return {
display: detail.display || base.display,
free: Boolean(detail.free || base.free),
prime: Boolean(detail.prime || base.prime),
fastestDate: detail.fastestDate ?? base.fastestDate
};
}
export function extractDetailPage(html: string, base: ProductSearchResult): ProductSearchResult {
const root = parse(html);
const title = firstText(root, ["#productTitle", "h1"]) || base.title;
const priceText = firstText(root, [
"#corePriceDisplay_desktop_feature_div .a-offscreen",
".a-price .a-offscreen",
".a-price"
]);
const deliveryText = firstText(root, [
"#mir-layout-DELIVERY_BLOCK-slot-PRIMARY_DELIVERY_MESSAGE_LARGE",
"#deliveryBlockMessage",
"[data-csa-c-delivery-price]"
]);
const availability = firstText(root, ["#availability", "#availabilityInsideBuyBox_feature_div"]);
const seller = firstText(root, ["#merchant-info", "#sellerProfileTriggerId"]);
const ratingText = attrOf(root.querySelector("#acrPopover"), "title") || textOf(root.querySelector("#acrPopover"));
const reviewText = textOf(root.querySelector("#acrCustomerReviewText"));
const histogram = parseStarBreakdown(extractHistogramText(root));
const product: ProductSearchResult = {
...base,
title,
price: parseMoney(priceText) ?? base.price,
rating: parseRating(ratingText) ?? base.rating,
reviewCount: parseReviewCount(reviewText) ?? base.reviewCount,
delivery: mergeDelivery(deliveryFromText(deliveryText, hasPrimeSignal(root)), base.delivery),
availability: availability || base.availability,
seller: seller || base.seller,
bullets: extractBullets(root),
specs: extractSpecs(root),
starBreakdown: histogram ?? base.starBreakdown,
missingFields: [...base.missingFields],
extractionNotes: [...base.extractionNotes]
};
for (const field of ["price", "delivery", "rating", "reviewCount", "starBreakdown"] as const) {
if (product[field] === undefined && !product.missingFields.includes(field)) {
product.missingFields.push(field);
}
}
return product;
}
+123
View File
@@ -0,0 +1,123 @@
import type { FilteredProducts, ProductFilters, ProductSearchResult } from "./types.js";
import { extractWidthInches } from "./product-metrics.js";
function passesMin(value: number | undefined, threshold: number, comparison: "gt" | "gte" | undefined): boolean {
if (value === undefined) {
return false;
}
return comparison === "gt" ? value > threshold : value >= threshold;
}
function filterReasons(product: ProductSearchResult, filters: ProductFilters): string[] {
const reasons: string[] = [];
if (filters.minRating !== undefined && !passesMin(product.rating, filters.minRating, filters.ratingComparison)) {
reasons.push(product.rating === undefined ? "rating unknown" : `rating ${product.rating} below filter`);
}
if (filters.minReviews !== undefined && !passesMin(product.reviewCount, filters.minReviews, filters.reviewCountComparison)) {
reasons.push(product.reviewCount === undefined ? "review count unknown" : `review count ${product.reviewCount} below filter`);
}
if (filters.maxPrice !== undefined) {
if (!product.price) {
reasons.push("price unknown");
} else if (product.price.amount > filters.maxPrice) {
reasons.push(`price ${product.price.display} above filter`);
}
}
if (filters.maxUnitPrice !== undefined) {
if (!product.unitPrice) {
reasons.push("unit price unknown");
} else if (product.unitPrice.amount > filters.maxUnitPrice) {
reasons.push(`unit price ${product.unitPrice.display} above filter`);
}
}
if (filters.minWidthInches !== undefined) {
const width = extractWidthInches(product);
if (width === undefined) {
reasons.push("width unknown");
} else if (!passesMin(width, filters.minWidthInches, filters.widthComparison)) {
reasons.push(`width ${width} inches below filter`);
}
}
if (filters.requirePrime && !product.delivery?.prime) {
reasons.push("Prime delivery not verified");
}
if (filters.requireFreeDelivery && !product.delivery?.free) {
reasons.push("free delivery not verified");
}
if (filters.deliveryBy && !deliveryMatches(product.delivery?.display, filters.deliveryBy)) {
reasons.push(`${filters.deliveryBy} delivery not verified`);
}
return reasons;
}
function deliveryMatches(display: string | undefined, deliveryBy: string): boolean {
if (!display) {
return false;
}
const normalized = display.toLowerCase();
if (deliveryBy === "today") {
return /\btoday\b|same[- ]day/.test(normalized);
}
if (deliveryBy === "tomorrow" || deliveryBy === "overnight") {
return /\btomorrow\b|overnight|next[- ]day|one[- ]day/.test(normalized);
}
return normalized.includes(deliveryBy.toLowerCase());
}
function comparisonSymbol(comparison: "gt" | "gte" | undefined): string {
return comparison === "gt" ? ">" : ">=";
}
function rankProducts(a: ProductSearchResult, b: ProductSearchResult, filters: ProductFilters): number {
if (filters.sortBy === "price") {
return (a.price?.amount ?? Number.POSITIVE_INFINITY) - (b.price?.amount ?? Number.POSITIVE_INFINITY);
}
const ratingDiff = (b.rating ?? -1) - (a.rating ?? -1);
if (ratingDiff !== 0) return ratingDiff;
const reviewDiff = (b.reviewCount ?? -1) - (a.reviewCount ?? -1);
if (reviewDiff !== 0) return reviewDiff;
return (a.price?.amount ?? Number.POSITIVE_INFINITY) - (b.price?.amount ?? Number.POSITIVE_INFINITY);
}
export function applyFiltersAndLimit(
products: ProductSearchResult[],
filters: ProductFilters,
limit: number
): FilteredProducts {
const filteredOutReasons: Record<string, string[]> = {};
const uniqueProducts = new Map<string, ProductSearchResult>();
for (const product of products) {
if (!uniqueProducts.has(product.asin)) {
uniqueProducts.set(product.asin, product);
}
}
const passing: ProductSearchResult[] = [];
for (const product of uniqueProducts.values()) {
const reasons = filterReasons(product, filters);
if (reasons.length > 0) {
filteredOutReasons[product.asin] = reasons;
continue;
}
passing.push({
...product,
matchedFilters: [
...product.matchedFilters,
...(filters.minRating !== undefined ? [`rating ${filters.ratingComparison ?? "gte"} ${filters.minRating}`] : []),
...(filters.minReviews !== undefined ? [`reviews ${filters.reviewCountComparison ?? "gte"} ${filters.minReviews}`] : []),
...(filters.maxPrice !== undefined ? [`price <= ${filters.maxPrice}`] : []),
...(filters.maxUnitPrice !== undefined ? [`unit price <= ${filters.maxUnitPrice}`] : []),
...(filters.minWidthInches !== undefined ? [`width ${comparisonSymbol(filters.widthComparison)} ${filters.minWidthInches} inches`] : []),
...(filters.requirePrime ? ["Prime delivery"] : []),
...(filters.requireFreeDelivery ? ["free delivery"] : []),
...(filters.deliveryBy ? [`delivery by ${filters.deliveryBy}`] : [])
]
});
}
return {
results: passing.sort((a, b) => rankProducts(a, b, filters)).slice(0, limit),
filteredOutCount: uniqueProducts.size - passing.length,
filteredOutReasons
};
}
+118
View File
@@ -0,0 +1,118 @@
import type { MoneyValue, StarBreakdown, UnitCountExtraction } from "./types.js";
export function parseMoney(text: string | undefined | null): MoneyValue | undefined {
if (!text) {
return undefined;
}
const compact = text.replace(/\s+/g, " ").trim();
const match = compact.match(/\$\s*([0-9][0-9,]*(?:\.[0-9]{1,2})?)/);
if (!match) {
return undefined;
}
const amount = Number(match[1].replace(/,/g, ""));
if (!Number.isFinite(amount)) {
return undefined;
}
return {
amount,
currency: "USD",
display: compact
};
}
export function parseUnitPrice(text: string | undefined | null): MoneyValue | undefined {
if (!text) {
return undefined;
}
const compact = text.replace(/\s+/g, " ").trim();
const unitMatch = compact.match(/(\$\s*[0-9][0-9,]*(?:\.[0-9]{1,2})?)(?:\s*\$\s*[0-9][0-9,]*(?:\.[0-9]{1,2})?)?\s*(?:\/|\bper\b\s*)\s*(?:count|unit|item|piece|pack|each)\b/i)
?? compact.match(/(\$\s*[0-9][0-9,]*(?:\.[0-9]{1,2})?)\s*(?:each)\b/i);
if (!unitMatch) {
return undefined;
}
const display = unitMatch[0]
.replace(/\$\s*([0-9][0-9,]*(?:\.[0-9]{1,2})?)\s*\$\s*\1/i, "$$$1")
.replace(/\s+/g, "");
return parseMoney(display);
}
export function parseRating(text: string | undefined | null): number | undefined {
if (!text) {
return undefined;
}
const match = text.match(/([0-5](?:\.[0-9])?)\s*(?:out of\s*)?5\s*stars?/i)
?? text.match(/\brated\s+([0-5](?:\.[0-9])?)/i);
if (!match) {
return undefined;
}
const rating = Number(match[1]);
return Number.isFinite(rating) ? rating : undefined;
}
export function parseReviewCount(text: string | undefined | null): number | undefined {
if (!text) {
return undefined;
}
const match = text.match(/([0-9][0-9,]*)\s*(?:ratings?|reviews?)/i);
if (!match) {
return undefined;
}
const count = Number(match[1].replace(/,/g, ""));
return Number.isInteger(count) ? count : undefined;
}
export function parseStarBreakdown(text: string | undefined | null): StarBreakdown | undefined {
if (!text) {
return undefined;
}
const breakdown: Partial<Omit<StarBreakdown, "basis">> = {};
const words: Record<string, keyof Omit<StarBreakdown, "basis">> = {
"5": "five",
"4": "four",
"3": "three",
"2": "two",
"1": "one"
};
const percentMatches = [...text.matchAll(/([1-5])\s*star\s*([0-9]{1,3})\s*%/gi)];
if (percentMatches.length === 0) {
return undefined;
}
for (const match of percentMatches) {
const key = words[match[1]];
if (key) {
breakdown[key] = Number(match[2]);
}
}
return {
...breakdown,
basis: "percent"
};
}
export function extractUnitCount(text: string | undefined | null): UnitCountExtraction | undefined {
if (!text) {
return undefined;
}
const patterns = [
{ pattern: /(\d{1,4})\s*[- ]?(?:count|ct)\b/i, confidence: "high" as const },
{ pattern: /\bpack\s+of\s+(\d{1,4})\b/i, confidence: "high" as const },
{ pattern: /\b(\d{1,4})\s*[- ]?pack\b/i, confidence: "high" as const },
{ pattern: /\bset\s+of\s+(\d{1,4})\b/i, confidence: "medium" as const },
{ pattern: /\b(\d{1,4})\s+(?:bulbs?|cables?|pieces?|pcs)\b/i, confidence: "low" as const }
];
for (const { pattern, confidence } of patterns) {
const match = text.match(pattern);
if (!match) {
continue;
}
const count = Number(match[1]);
if (Number.isInteger(count) && count > 0) {
return {
count,
confidence,
source: match[0]
};
}
}
return undefined;
}
@@ -0,0 +1,62 @@
import type { ProductSearchResult, ProductSpec } from "./types.js";
function parseDimensionNumber(text: string): number | undefined {
const match = text.match(/([0-9]+(?:\.[0-9]+)?)/);
return match ? Number(match[1]) : undefined;
}
function isOverallWidthSpec(spec: ProductSpec): boolean {
const name = spec.name.toLowerCase();
if (/seat|arm|door|package|box|back|cushion/.test(name)) {
return false;
}
return /width|dimensions?/.test(name);
}
function widthFromSpec(spec: ProductSpec): number | undefined {
if (!isOverallWidthSpec(spec)) {
return undefined;
}
const name = spec.name.toLowerCase();
const value = spec.value;
const labeledWidth = value.match(/([0-9]+(?:\.[0-9]+)?)\s*(?:"|in(?:ches?)?)?\s*W\b/i);
if (labeledWidth) {
return Number(labeledWidth[1]);
}
if (/width/.test(name)) {
return parseDimensionNumber(value);
}
const orderMatch = name.match(/\b([dwh])\s*x\s*([dwh])(?:\s*x\s*([dwh]))?\b/i);
if (orderMatch) {
const order = orderMatch.slice(1).filter(Boolean).map((part) => part.toLowerCase());
const widthIndex = order.indexOf("w");
const values = value.match(/[0-9]+(?:\.[0-9]+)?/g)?.map(Number) ?? [];
if (widthIndex >= 0 && values[widthIndex] !== undefined) {
return values[widthIndex];
}
}
return undefined;
}
export function extractWidthInches(product: ProductSearchResult): number | undefined {
for (const spec of product.specs) {
const width = widthFromSpec(spec);
if (width !== undefined) {
return width;
}
}
const titleMatch = product.title.match(/\b([0-9]+(?:\.[0-9]+)?)\s*(?:["”]|in(?:ch(?:es)?)?)\b/i);
return titleMatch ? Number(titleMatch[1]) : undefined;
}
export function formatWidthInches(width: number | undefined): string {
if (width === undefined) {
return "unknown";
}
return `${Number.isInteger(width) ? width.toFixed(0) : width.toFixed(1)}"`;
}
+111
View File
@@ -0,0 +1,111 @@
import type { ParsedNaturalLanguageRequest, ProductFilters } from "./types.js";
function cleanQuery(text: string): string {
return text
.replace(/\breview score of\b/gi, " ")
.replace(/\brating of\b/gi, " ")
.replace(/\b(?:delivery|shipping)\s+only\b/gi, " ")
.replace(/\blow\s+to\s+high\b/gi, " ")
.replace(/\bhigh\s+to\s+low\b/gi, " ")
.replace(/\bof\s+in\s+width\b/gi, " ")
.replace(/\bin\s+width\b/gi, " ")
.replace(/\b(?:that|and|with|have)\b/gi, " ")
.replace(/[,\s]+/g, " ")
.replace(/\s+/g, " ")
.replace(/\s+(and|or|a)$/i, "")
.trim();
}
function removeMatched(text: string, match: RegExpMatchArray | null): string {
if (!match) {
return text;
}
return text.replace(match[0], " ");
}
export function parseNaturalLanguageRequest(input: string): ParsedNaturalLanguageRequest {
let remaining = input.trim();
const filters: ProductFilters = {
includeKeywords: [],
excludeKeywords: []
};
let limit: number | undefined;
const limitMatch = remaining.match(/\b(?:return|limit|top)\s+(\d{1,3})\b/i);
if (limitMatch) {
limit = Number(limitMatch[1]);
remaining = removeMatched(remaining, limitMatch);
}
const sortByPriceMatch = remaining.match(/\b(?:by price|sort(?:ed)? by price|lowest price|cheapest|least expensive)\b/i);
if (sortByPriceMatch) {
filters.sortBy = "price";
remaining = removeMatched(remaining, sortByPriceMatch);
}
const deliveryTomorrowMatch = remaining.match(/\b(?:delivery|delivered|arrives?|shipping|ships?)\s+(?:by\s+)?tomorrow\b/i);
const deliveryTodayMatch = remaining.match(/\b(?:delivery|delivered|arrives?|shipping|ships?)\s+(?:by\s+)?today\b/i)
?? remaining.match(/\bsame[- ]day\s+(?:delivery|shipping)\b/i);
const overnightMatch = remaining.match(/\bovernight\s+(?:delivery|shipping)\b/i)
?? remaining.match(/\bnext[- ]day\s+(?:delivery|shipping)\b/i);
const deliveryMatch = overnightMatch ?? deliveryTomorrowMatch ?? deliveryTodayMatch;
if (deliveryMatch) {
filters.deliveryBy = overnightMatch ? "overnight" : deliveryTomorrowMatch ? "tomorrow" : "today";
remaining = removeMatched(remaining, deliveryMatch);
}
const primeMatch = remaining.match(/\b(?:(?:shipped|ships|shipping|delivery|delivered)\s+(?:with|by|from)\s+)?prime\b/i);
if (primeMatch) {
filters.requirePrime = true;
remaining = removeMatched(remaining, primeMatch);
}
const widthMatch = remaining.match(/\b(?:width\s*(?:of\s*)?)?([0-9]+(?:\.[0-9]+)?)\s*(?:inches|inch|in\.?|")\s*(?:or\s+)?(?:wider|wide|larger|longer)\b/i)
?? remaining.match(/\b([0-9]+(?:\.[0-9]+)?)\s*(?:inches|inch|in\.?|")\s*(?:or\s+)?(?:wider|wide|larger|longer)\s+(?:in\s+)?width\b/i)
?? remaining.match(/\b(?:at least|minimum|min\.?)\s+([0-9]+(?:\.[0-9]+)?)\s*(?:inches|inch|in\.?|")\s+(?:wide|width)\b/i);
if (widthMatch) {
filters.minWidthInches = Number(widthMatch[1]);
filters.widthComparison = "gte";
remaining = removeMatched(remaining, widthMatch);
}
const unitPriceMatch = remaining.match(/\b(?:cost\s+)?(?:less than|under|below)\s+\$([0-9]+(?:\.[0-9]{1,2})?)\s*(?:each|per\b|\/\s*(?:count|unit|item))\b/i);
if (unitPriceMatch) {
filters.maxUnitPrice = Number(unitPriceMatch[1]);
remaining = removeMatched(remaining, unitPriceMatch);
}
const maxPriceMatch = remaining.match(/\b(?:cost\s+)?(?:less than|under|below)\s+\$([0-9]+(?:\.[0-9]{1,2})?)\b/i);
if (maxPriceMatch) {
filters.maxPrice = Number(maxPriceMatch[1]);
remaining = removeMatched(remaining, maxPriceMatch);
}
const exclusiveReviews = remaining.match(/\b(?:over|more than|above)\s+([0-9][0-9,]*)\s*(?:reviews?|ratings?)\b/i);
const inclusiveReviews = remaining.match(/\b(?:at least|minimum|min\.?)\s+([0-9][0-9,]*)\s*(?:reviews?|ratings?)\b/i)
?? remaining.match(/\b([0-9][0-9,]*)\s*\+\s*(?:reviews?|ratings?)\b/i);
const reviewMatch = exclusiveReviews ?? inclusiveReviews;
if (reviewMatch) {
filters.minReviews = Number(reviewMatch[1].replace(/,/g, ""));
filters.reviewCountComparison = exclusiveReviews ? "gt" : "gte";
remaining = removeMatched(remaining, reviewMatch);
}
const exclusiveRating = remaining.match(/\b(?:a\s+)?(?:(?:review score|rating)\s+of\s+|rating\s+)?(?:more than|over|above|rated above)\s+([0-5](?:\.[0-9])?)\s*(?:stars?)?\b/i);
const inclusiveRating = remaining.match(/\b(?:a\s+)?(?:review score|rating)(?:\s+of)?\s+([0-5](?:\.[0-9])?)\s*(?:stars?)?\s+(?:or|and)\s+(?:higher|better)\b/i)
?? remaining.match(/\b([0-5](?:\.[0-9])?)\s*stars?\s+or\s+better\b/i)
?? remaining.match(/\b([0-5](?:\.[0-9])?)\s*stars?\s+(?:or|and)\s+(?:higher|better)\b/i)
?? remaining.match(/\b(?:at least|minimum|min\.?)\s+([0-5](?:\.[0-9])?)\s*(?:stars?|rating)?\b/i);
const ratingMatch = exclusiveRating ?? inclusiveRating;
if (ratingMatch) {
filters.minRating = Number(ratingMatch[1]);
filters.ratingComparison = exclusiveRating ? "gt" : "gte";
remaining = removeMatched(remaining, ratingMatch);
}
return {
query: cleanQuery(remaining),
filters,
limit
};
}
+131
View File
@@ -0,0 +1,131 @@
import type { ProductFilters, ProductSearchResult, SearchProductsResponse } from "./types.js";
import { extractWidthInches, formatWidthInches } from "./product-metrics.js";
export interface ResponseInput {
query: string;
filters: ProductFilters;
limit: number;
maxSearchPages: number;
results: ProductSearchResult[];
filteredOutCount: number;
warnings: string[];
now?: () => Date;
}
export function createResponse(input: ResponseInput): SearchProductsResponse {
return {
query: input.query,
filters: input.filters,
limit: input.limit,
maxSearchPages: input.maxSearchPages,
results: input.results,
filteredOutCount: input.filteredOutCount,
warnings: input.warnings,
source: {
site: "amazon.com",
scrapedAt: (input.now ?? (() => new Date()))().toISOString(),
automation: "web-automation/CloakBrowser"
}
};
}
function formatFilters(filters: ProductFilters): string {
const parts = [
filters.minRating !== undefined ? `rating ${filters.ratingComparison ?? "gte"} ${filters.minRating}` : "",
filters.minReviews !== undefined ? `reviews ${filters.reviewCountComparison ?? "gte"} ${filters.minReviews}` : "",
filters.maxPrice !== undefined ? `price <= $${filters.maxPrice}` : "",
filters.maxUnitPrice !== undefined ? `unit price <= $${filters.maxUnitPrice}` : "",
filters.minWidthInches !== undefined ? `width ${filters.widthComparison ?? "gte"} ${filters.minWidthInches} inches` : "",
filters.requirePrime ? "Prime delivery" : "",
filters.requireFreeDelivery ? "free delivery" : "",
filters.deliveryBy ? `delivery by ${filters.deliveryBy}` : "",
filters.sortBy === "price" ? "sort by price" : ""
].filter(Boolean);
return parts.length > 0 ? parts.join(", ") : "none";
}
function compactText(value: string): string {
return value.replace(/\s+/g, " ").trim();
}
function marker(passes: boolean | undefined, enabled: boolean): string {
if (!enabled) {
return "";
}
return passes ? " OK" : " NO";
}
function widthCell(product: ProductSearchResult, filters: ProductFilters): string {
const width = extractWidthInches(product);
const passes = width !== undefined && (filters.widthComparison === "gt" ? width > (filters.minWidthInches ?? 0) : width >= (filters.minWidthInches ?? 0));
return `${formatWidthInches(width)}${marker(passes, filters.minWidthInches !== undefined)}`;
}
function primeCell(product: ProductSearchResult, filters: ProductFilters): string {
if (product.delivery?.prime) {
return `Prime${marker(true, Boolean(filters.requirePrime))}`;
}
return `not verified${marker(false, Boolean(filters.requirePrime))}`;
}
function deliveryCell(product: ProductSearchResult, filters: ProductFilters): string {
const display = product.delivery?.display ?? "unknown";
if (!filters.deliveryBy) {
return display;
}
const normalized = display.toLowerCase();
const passes = filters.deliveryBy === "today"
? /\btoday\b|same[- ]day/.test(normalized)
: filters.deliveryBy === "tomorrow" || filters.deliveryBy === "overnight"
? /\btomorrow\b|overnight|next[- ]day|one[- ]day/.test(normalized)
: normalized.includes(filters.deliveryBy.toLowerCase());
return `${display}${marker(passes, true)}`;
}
function resultBlocks(products: ProductSearchResult[], filters: ProductFilters): string[] {
return products.flatMap((product, index) => [
`${index + 1}. ${compactText(product.title)}`,
`Price: ${product.price?.display ?? "unknown"}`,
`Rating: ${product.rating ?? "unknown"} stars`,
`Reviews: ${product.reviewCount?.toLocaleString("en-US") ?? "unknown"}`,
`Width: ${widthCell(product, filters)}`,
`Prime: ${primeCell(product, filters)}`,
`Delivery: ${compactText(deliveryCell(product, filters))}`,
`Link: ${product.url}`,
""
]);
}
function metadataLines(products: ProductSearchResult[]): string[] {
const lines: string[] = [];
for (const product of products) {
const notes = [
product.missingFields.length > 0 ? `missing ${product.missingFields.join(", ")}` : "",
product.isSponsored ? "sponsored" : "",
product.extractionNotes.length > 0 ? product.extractionNotes.join("; ") : ""
].filter(Boolean);
if (notes.length > 0) {
lines.push(`- ${product.title}: ${notes.join("; ")}`);
}
}
return lines;
}
export function createMarkdownReport(response: SearchProductsResponse): string {
const lines = [
`# Amazon Shopping Results`,
"",
`Query: ${response.query}`,
`Filters: ${formatFilters(response.filters)}`,
`Results returned: ${response.results.length} (filtered out: ${response.filteredOutCount})`,
response.warnings.length > 0 ? `Warnings: ${response.warnings.join("; ")}` : "",
"",
"## Best Matches",
"",
response.results.length > 0 ? "" : "No products matched all requested filters.",
...resultBlocks(response.results, response.filters),
"",
...metadataLines(response.results)
].filter((line) => line !== "");
return `${lines.join("\n")}\n`;
}
+155
View File
@@ -0,0 +1,155 @@
import { HTMLElement, parse } from "node-html-parser";
import { parseMoney, parseRating, parseReviewCount, parseUnitPrice } from "./parsers.js";
import type { DeliverySummary, ProductSearchResult, SearchPageExtraction } from "./types.js";
function textOf(node: HTMLElement | null | undefined): string {
return node?.textContent.replace(/\s+/g, " ").trim() ?? "";
}
function attrOf(node: HTMLElement | null | undefined, name: string): string | undefined {
return node?.getAttribute(name) ?? undefined;
}
function absoluteAmazonUrl(href: string | undefined, currentUrl = "https://www.amazon.com/"): string | undefined {
if (!href) {
return undefined;
}
if (href.startsWith("https://www.amazon.com")) {
return href;
}
try {
const parsed = new URL(href, currentUrl);
if (parsed.hostname !== "www.amazon.com") {
return undefined;
}
return parsed.toString();
} catch {
return undefined;
}
}
function normalizeProductUrl(asin: string, href: string | undefined, currentUrl: string): string {
const absolute = absoluteAmazonUrl(href, currentUrl);
if (!absolute) {
return `https://www.amazon.com/dp/${asin}`;
}
try {
const url = new URL(absolute);
const match = url.pathname.match(/\/(?:dp|gp\/product)\/([A-Z0-9]{8,14})/i);
if (match) {
return `https://www.amazon.com/dp/${match[1].toUpperCase()}`;
}
} catch {
return `https://www.amazon.com/dp/${asin}`;
}
return `https://www.amazon.com/dp/${asin}`;
}
function detectChallenge(html: string): boolean {
return /robot check|enter the characters you see|captcha|automated access|access denied/i.test(html);
}
function deliveryFromText(text: string, primeSignal = false): DeliverySummary | undefined {
const compact = text.replace(/\s+/g, " ").trim();
const deliveryMatch = compact.match(/((?:FREE\s+)?delivery\b[^.]+)/i);
if (!deliveryMatch) {
return primeSignal ? { display: "Prime delivery available", prime: true } : undefined;
}
const display = deliveryMatch[1].trim();
return {
display,
free: /\bfree\b/i.test(display),
prime: primeSignal || /\bprime\b/i.test(display)
};
}
function hasPrimeSignal(card: HTMLElement): boolean {
const attributeText = card.querySelectorAll("[class], [aria-label], img[alt]")
.map((node) => [
attrOf(node, "class") ?? "",
attrOf(node, "aria-label") ?? "",
attrOf(node, "alt") ?? ""
].join(" "))
.join(" ");
return /a-icon-prime|prime-logo|amazon\s+prime|\bprime\b/i.test(attributeText);
}
function firstText(card: HTMLElement, selectors: string[]): string {
for (const selector of selectors) {
const value = textOf(card.querySelector(selector));
if (value) {
return value;
}
}
return "";
}
function firstUnitPriceText(card: HTMLElement): string {
for (const node of card.querySelectorAll(".a-color-secondary, .a-size-base, span")) {
const value = textOf(node);
if (parseUnitPrice(value)) {
return value;
}
}
return "";
}
export function extractSearchPage(html: string, currentUrl: string): SearchPageExtraction {
if (detectChallenge(html)) {
return {
status: "challenge",
products: [],
warnings: ["Amazon returned a challenge or blocked page; stopping without bypass."],
};
}
const root = parse(html);
const cards = root.querySelectorAll("[data-asin]")
.filter((card) => /^[A-Z0-9]{8,14}$/i.test(card.getAttribute("data-asin") ?? ""));
const products: ProductSearchResult[] = [];
for (const card of cards) {
const asin = (card.getAttribute("data-asin") ?? "").toUpperCase();
const link = card.querySelector("h2 a") ?? card.querySelector("a[href*='/dp/']") ?? card.querySelector("a[href*='/gp/product/']");
const title = textOf(link) || firstText(card, ["h2", "[data-cy='title-recipe']"]);
if (!title) {
continue;
}
const priceText = firstText(card, [".a-price .a-offscreen", ".a-price"]);
const allText = textOf(card);
const unitPriceText = firstUnitPriceText(card);
const ariaText = card.querySelectorAll("[aria-label]")
.map((node) => attrOf(node, "aria-label") ?? "")
.join(" ");
const delivery = deliveryFromText(allText, hasPrimeSignal(card));
const product: ProductSearchResult = {
asin,
title,
url: normalizeProductUrl(asin, attrOf(link, "href"), currentUrl),
imageUrl: attrOf(card.querySelector("img"), "src"),
price: parseMoney(priceText),
unitPrice: parseUnitPrice(unitPriceText),
rating: parseRating(ariaText || allText),
reviewCount: parseReviewCount(ariaText || allText),
delivery,
specs: [],
bullets: [],
isSponsored: /\bsponsored\b/i.test(allText),
matchedFilters: [],
missingFields: [],
extractionNotes: []
};
products.push(product);
}
const nextHref = attrOf(root.querySelector(".s-pagination-next[href]"), "href");
const nextPageUrl = absoluteAmazonUrl(nextHref, currentUrl);
return {
status: "ok",
products,
warnings: [],
nextPageUrl: nextPageUrl ?? undefined
};
}
+114
View File
@@ -0,0 +1,114 @@
export interface SearchProductsRequest {
query: string;
filters: ProductFilters;
limit: number;
maxSearchPages: number;
skipDetails: boolean;
dryRun: boolean;
output: "json" | "markdown" | "both";
}
export interface ProductFilters {
minRating?: number;
ratingComparison?: "gt" | "gte";
minReviews?: number;
reviewCountComparison?: "gt" | "gte";
maxPrice?: number;
maxUnitPrice?: number;
minWidthInches?: number;
widthComparison?: "gt" | "gte";
includeKeywords: string[];
excludeKeywords: string[];
requirePrime?: boolean;
requireFreeDelivery?: boolean;
deliveryBy?: string;
sortBy?: "relevance" | "price";
}
export interface ProductSearchResult {
asin: string;
title: string;
url: string;
imageUrl?: string;
price?: MoneyValue;
unitPrice?: MoneyValue;
rating?: number;
reviewCount?: number;
starBreakdown?: StarBreakdown;
delivery?: DeliverySummary;
specs: ProductSpec[];
bullets: string[];
seller?: string;
isSponsored?: boolean;
availability?: string;
matchedFilters: string[];
missingFields: string[];
extractionNotes: string[];
}
export interface MoneyValue {
amount: number;
currency: "USD";
display: string;
}
export interface DeliverySummary {
display: string;
prime?: boolean;
free?: boolean;
fastestDate?: string;
}
export interface StarBreakdown {
five?: number;
four?: number;
three?: number;
two?: number;
one?: number;
basis: "percent" | "count";
}
export interface ProductSpec {
name: string;
value: string;
}
export interface SearchProductsResponse {
query: string;
filters: ProductFilters;
limit: number;
maxSearchPages: number;
results: ProductSearchResult[];
filteredOutCount: number;
warnings: string[];
source: {
site: "amazon.com";
scrapedAt: string;
automation: "web-automation/CloakBrowser";
};
}
export interface ParsedNaturalLanguageRequest {
query: string;
filters: ProductFilters;
limit?: number;
}
export interface UnitCountExtraction {
count: number;
confidence: "high" | "medium" | "low";
source: string;
}
export interface SearchPageExtraction {
status: "ok" | "challenge";
products: ProductSearchResult[];
warnings: string[];
nextPageUrl?: string;
}
export interface FilteredProducts {
results: ProductSearchResult[];
filteredOutCount: number;
filteredOutReasons: Record<string, string[]>;
}
@@ -0,0 +1,71 @@
import { access } from "node:fs/promises";
import { constants } from "node:fs";
import { dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
export interface RuntimeResolverOptions {
env?: NodeJS.ProcessEnv;
homeDir?: string;
skillDir?: string;
}
export interface WebAutomationRuntime {
scriptsDir: string;
checkInstall: {
cwd: string;
command: string;
args: string[];
};
}
async function assertFile(path: string, label: string): Promise<void> {
try {
await access(path, constants.F_OK);
} catch {
throw new Error(`web-automation runtime is missing ${label}: ${path}`);
}
}
async function assertExecutableOrFile(path: string, label: string): Promise<void> {
try {
await access(path, constants.X_OK);
} catch {
await assertFile(path, label);
}
}
function defaultSkillDir(): string {
return resolve(dirname(fileURLToPath(import.meta.url)), "..");
}
export async function resolveWebAutomationRuntime(options: RuntimeResolverOptions = {}): Promise<WebAutomationRuntime> {
const env = options.env ?? process.env;
const homeDir = options.homeDir ?? process.env.HOME ?? "";
const skillDir = options.skillDir ?? defaultSkillDir();
const candidates = [
env.AMAZON_SHOPPING_WEB_AUTOMATION_DIR,
homeDir ? join(homeDir, ".openclaw", "workspace", "skills", "web-automation", "scripts") : undefined,
resolve(skillDir, "..", "web-automation", "scripts")
].filter((candidate): candidate is string => Boolean(candidate));
const errors: string[] = [];
for (const scriptsDir of candidates) {
try {
await assertFile(join(scriptsDir, "check-install.js"), "check-install.js");
await assertFile(join(scriptsDir, "package.json"), "package.json");
await assertExecutableOrFile(join(scriptsDir, "node_modules", ".bin", "tsx"), "node_modules/.bin/tsx");
return {
scriptsDir,
checkInstall: {
cwd: scriptsDir,
command: "node",
args: ["check-install.js"]
}
};
} catch (error: unknown) {
errors.push(error instanceof Error ? error.message : String(error));
}
}
throw new Error(`Unable to locate usable web-automation runtime.\n${errors.join("\n")}`);
}
@@ -0,0 +1,43 @@
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import { classifyHttpStatus, isPathAllowedByRobots, plannedAmazonPaths } from "../src/browser.js";
describe("browser compliance helpers", () => {
it("plans only search and product-detail paths", () => {
assert.deepEqual(plannedAmazonPaths(["B0TEST0001"]), ["/s", "/dp/B0TEST0001", "/gp/product/B0TEST0001"]);
});
it("honors robots disallow rules for planned paths", () => {
const robots = `
User-agent: *
Disallow: /cart
Disallow: /product-reviews
Disallow: /dp/private
`;
assert.equal(isPathAllowedByRobots(robots, "*", "/s"), true);
assert.equal(isPathAllowedByRobots(robots, "*", "/product-reviews/B0TEST0001"), false);
assert.equal(isPathAllowedByRobots(robots, "*", "/dp/private/B0TEST0001"), false);
});
it("does not leak disallow rules from other user-agent groups", () => {
const robots = `
User-agent: specialbot
Disallow: /dp
User-agent: *
Disallow: /cart
`;
assert.equal(isPathAllowedByRobots(robots, "*", "/dp/B0TEST0001"), true);
assert.equal(isPathAllowedByRobots(robots, "specialbot", "/dp/B0TEST0001"), false);
});
it("classifies retryable and challenge statuses", () => {
assert.equal(classifyHttpStatus(429), "retryable");
assert.equal(classifyHttpStatus(503), "retryable");
assert.equal(classifyHttpStatus(403), "challenge");
assert.equal(classifyHttpStatus(200), "ok");
});
});
+132
View File
@@ -0,0 +1,132 @@
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import { buildSearchUrl, parseCliRequest, runCli } from "../src/cli.js";
function createOutput() {
let stdout = "";
let stderr = "";
return {
stdout: { write: (chunk: string) => { stdout += chunk; return true; } },
stderr: { write: (chunk: string) => { stderr += chunk; return true; } },
get stdoutText() { return stdout; },
get stderrText() { return stderr; }
};
}
describe("amazon-shopping CLI", () => {
it("prints help", async () => {
const output = createOutput();
const code = await runCli(["--help"], output);
assert.equal(code, 0);
assert.match(output.stdoutText, /scripts\/search-products/);
assert.match(output.stdoutText, /--dry-run/);
});
it("defaults to 15 results and two search pages", () => {
const request = parseCliRequest(["usb c cable"]);
assert.equal(request.query, "usb c cable");
assert.equal(request.limit, 15);
assert.equal(request.maxSearchPages, 2);
assert.equal(request.output, "json");
});
it("maps kebab-case CLI filters into the request contract", () => {
const request = parseCliRequest([
"--query",
"100w led bulbs",
"--min-rating",
"4.5",
"--min-reviews",
"200",
"--max-unit-price",
"4",
"--min-width",
"77",
"--require-prime",
"--delivery-by",
"tomorrow",
"--sort-by",
"price",
"--max-search-pages",
"3",
"--skip-details",
"--dry-run"
]);
assert.equal(request.query, "100w led bulbs");
assert.equal(request.filters.minRating, 4.5);
assert.equal(request.filters.minReviews, 200);
assert.equal(request.filters.maxUnitPrice, 4);
assert.equal(request.filters.minWidthInches, 77);
assert.equal(request.filters.requirePrime, true);
assert.equal(request.filters.deliveryBy, "tomorrow");
assert.equal(request.filters.sortBy, "price");
assert.equal(request.maxSearchPages, 3);
assert.equal(request.skipDetails, true);
assert.equal(request.dryRun, true);
});
it("maps output modes", () => {
assert.equal(parseCliRequest(["usb c cable", "--json"]).output, "json");
assert.equal(parseCliRequest(["usb c cable", "--markdown"]).output, "markdown");
assert.equal(parseCliRequest(["usb c cable", "--json", "--markdown"]).output, "both");
});
it("accepts max as a natural agent alias for limit", () => {
const request = parseCliRequest(["100w led bulbs", "--max", "5"]);
assert.equal(request.limit, 5);
});
it("normalizes natural-language filters for the target request", () => {
const request = parseCliRequest([
"100w led bulbs that cost less than $4 each and have over 200 reviews with a review score of more than 4.5 stars",
"--dry-run"
]);
assert.equal(request.query, "100w led bulbs");
assert.equal(request.filters.maxUnitPrice, 4);
assert.equal(request.filters.minReviews, 200);
assert.equal(request.filters.reviewCountComparison, "gt");
assert.equal(request.filters.minRating, 4.5);
assert.equal(request.filters.ratingComparison, "gt");
});
it("rejects limits below one", () => {
assert.throws(
() => parseCliRequest(["usb c cable", "--limit", "0"]),
/limit must be an integer greater than 0/
);
});
it("rejects unsafe large limits unless explicitly allowed", () => {
assert.throws(
() => parseCliRequest(["usb c cable", "--limit", "31"]),
/require --allow-large-limit/
);
});
it("rejects search page caps above five", () => {
assert.throws(
() => parseCliRequest(["usb c cable", "--max-search-pages", "6"]),
/max-search-pages must be 5 or less/
);
});
it("rejects unsupported sort modes", () => {
assert.throws(
() => parseCliRequest(["usb c cable", "--sort-by", "rating"]),
/sort-by must be either price or relevance/
);
});
it("builds the Amazon search URL without live network access", () => {
assert.equal(
buildSearchUrl("100w led bulbs"),
"https://www.amazon.com/s?k=100w%20led%20bulbs"
);
});
});
@@ -0,0 +1,118 @@
import assert from "node:assert/strict";
import { readFile } from "node:fs/promises";
import { join } from "node:path";
import { describe, it } from "node:test";
import { extractDetailPage } from "../src/detail-page.js";
const fixturePath = join(import.meta.dirname, "fixtures", "product-detail.html");
describe("extractDetailPage", () => {
it("extracts visible product detail fields from sanitized HTML", async () => {
const html = await readFile(fixturePath, "utf8");
const details = extractDetailPage(html, {
asin: "B0TESTLED1",
title: "Search title",
url: "https://www.amazon.com/dp/B0TESTLED1",
specs: [],
bullets: [],
matchedFilters: [],
missingFields: [],
extractionNotes: []
});
assert.equal(details.title, "Bright Daylight LED Bulbs 100W Equivalent, 50 Count");
assert.equal(details.price?.amount, 18.99);
assert.equal(details.delivery?.free, true);
assert.equal(details.availability, "In Stock");
assert.equal(details.seller, "Ships from Amazon.com");
assert.equal(details.bullets.length, 2);
assert.deepEqual(details.specs[0], { name: "Brand", value: "BrightCo" });
assert.equal(details.rating, 4.6);
assert.equal(details.reviewCount, 1234);
assert.equal(details.starBreakdown?.five, 72);
});
it("records missing detail-only fields", () => {
const details = extractDetailPage("<html><body><h1 id=\"productTitle\">Sparse Product</h1></body></html>", {
asin: "B0SPARSE01",
title: "Sparse",
url: "https://www.amazon.com/dp/B0SPARSE01",
specs: [],
bullets: [],
matchedFilters: [],
missingFields: [],
extractionNotes: []
});
assert.equal(details.price, undefined);
assert.ok(details.missingFields.includes("price"));
assert.ok(details.missingFields.includes("starBreakdown"));
});
it("drops script-like spec rows and trims availability metadata", () => {
const details = extractDetailPage(`
<h1 id="productTitle">Messy Product</h1>
<div id="availability">In Stock {"merchantId":"secretish"}</div>
<table>
<tr><td>Special Feature</td><td>(function(P) { tracking(); }) Real feature text</td></tr>
<tr><td>A19 Add to Cart logShoppableMetrics("x", true)</td><td>Buying Options</td></tr>
<tr><td>Wattage</td><td>15 watts</td></tr>
<tr><td>Customer Reviews</td><td>4.7 out of 5 stars tracking payload</td></tr>
</table>
`, {
asin: "B0MESSY001",
title: "Messy",
url: "https://www.amazon.com/dp/B0MESSY001",
specs: [],
bullets: [],
matchedFilters: [],
missingFields: [],
extractionNotes: []
});
assert.equal(details.availability, "In Stock");
assert.deepEqual(details.specs, [{ name: "Wattage", value: "15 watts" }]);
});
it("preserves a search-card Prime signal when detail delivery text omits Prime", () => {
const details = extractDetailPage(`
<h1 id="productTitle">Prime Sofa Bed</h1>
<div id="mir-layout-DELIVERY_BLOCK-slot-PRIMARY_DELIVERY_MESSAGE_LARGE">FREE delivery Tomorrow. Details</div>
<table>
<tr><td>Product Dimensions</td><td>35"D x 83"W x 31"H</td></tr>
</table>
`, {
asin: "B0PRIME123",
title: "Prime Sofa Bed",
url: "https://www.amazon.com/dp/B0PRIME123",
delivery: { display: "FREE delivery Tomorrow", free: true, prime: true },
specs: [],
bullets: [],
matchedFilters: [],
missingFields: [],
extractionNotes: []
});
assert.equal(details.delivery?.prime, true);
assert.equal(details.delivery?.free, true);
});
it("does not treat Prime in a detail title as Prime delivery", () => {
const details = extractDetailPage(`
<h1 id="productTitle">Prime Sofa Bed</h1>
<div id="mir-layout-DELIVERY_BLOCK-slot-PRIMARY_DELIVERY_MESSAGE_LARGE">FREE delivery Tomorrow. Details</div>
`, {
asin: "B0TITLE123",
title: "Prime Sofa Bed",
url: "https://www.amazon.com/dp/B0TITLE123",
specs: [],
bullets: [],
matchedFilters: [],
missingFields: [],
extractionNotes: []
});
assert.equal(details.delivery?.prime, false);
});
});
@@ -0,0 +1,126 @@
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import { applyFiltersAndLimit } from "../src/filters.js";
import type { ProductSearchResult } from "../src/types.js";
function product(overrides: Partial<ProductSearchResult>): ProductSearchResult {
return {
asin: "B0BASE0001",
title: "Base Product",
url: "https://www.amazon.com/dp/B0BASE0001",
specs: [],
bullets: [],
matchedFilters: [],
missingFields: [],
extractionNotes: [],
...overrides
};
}
describe("applyFiltersAndLimit", () => {
it("applies strict rating, review, and unit-price filters", () => {
const result = applyFiltersAndLimit([
product({
asin: "B0PASS0001",
rating: 4.6,
reviewCount: 201,
unitPrice: { amount: 3.99, currency: "USD", display: "$3.99/Count" }
}),
product({
asin: "B0FAIL0001",
rating: 4.5,
reviewCount: 200,
unitPrice: { amount: 3.99, currency: "USD", display: "$3.99/Count" }
}),
product({
asin: "B0UNKNOWN1",
rating: 4.7,
reviewCount: 300
})
], {
includeKeywords: [],
excludeKeywords: [],
minRating: 4.5,
ratingComparison: "gt",
minReviews: 200,
reviewCountComparison: "gt",
maxUnitPrice: 4
}, 10);
assert.deepEqual(result.results.map((item) => item.asin), ["B0PASS0001"]);
assert.equal(result.filteredOutCount, 2);
assert.match(result.filteredOutReasons["B0UNKNOWN1"]?.join(" ") ?? "", /unit price unknown/i);
});
it("sorts by rating, reviews, then price", () => {
const result = applyFiltersAndLimit([
product({ asin: "B0LOWPRICE", rating: 4.7, reviewCount: 1000, price: { amount: 15, currency: "USD", display: "$15.00" } }),
product({ asin: "B0HIGHRATE", rating: 4.9, reviewCount: 100, price: { amount: 40, currency: "USD", display: "$40.00" } }),
product({ asin: "B0MOREREV", rating: 4.7, reviewCount: 2000, price: { amount: 20, currency: "USD", display: "$20.00" } })
], { includeKeywords: [], excludeKeywords: [] }, 2);
assert.deepEqual(result.results.map((item) => item.asin), ["B0HIGHRATE", "B0MOREREV"]);
});
it("deduplicates repeated ASINs before limiting", () => {
const result = applyFiltersAndLimit([
product({ asin: "B0DUP0001", rating: 4.8, reviewCount: 1000 }),
product({ asin: "B0DUP0001", rating: 4.8, reviewCount: 1000 }),
product({ asin: "B0UNIQUE1", rating: 4.7, reviewCount: 900 })
], { includeKeywords: [], excludeKeywords: [] }, 10);
assert.deepEqual(result.results.map((item) => item.asin), ["B0DUP0001", "B0UNIQUE1"]);
});
it("applies width, Prime, and delivery-by filters", () => {
const result = applyFiltersAndLimit([
product({
asin: "B0MATCH001",
rating: 4.3,
reviewCount: 250,
price: { amount: 399, currency: "USD", display: "$399.00" },
delivery: { display: "FREE delivery Tomorrow", free: true, prime: true },
specs: [{ name: "Product Dimensions", value: "35\"D x 83\"W x 31\"H" }]
}),
product({
asin: "B0NOPRIME1",
rating: 4.5,
reviewCount: 300,
delivery: { display: "FREE delivery Tomorrow", free: true, prime: false },
specs: [{ name: "Product Dimensions", value: "35\"D x 84\"W x 31\"H" }]
}),
product({
asin: "B0NARROW01",
rating: 4.6,
reviewCount: 300,
delivery: { display: "FREE delivery Tomorrow", free: true, prime: true },
specs: [{ name: "Product Dimensions", value: "35\"D x 65\"W x 31\"H" }]
})
], {
includeKeywords: [],
excludeKeywords: [],
minRating: 4,
minReviews: 200,
minWidthInches: 77,
requirePrime: true,
deliveryBy: "tomorrow"
}, 10);
assert.deepEqual(result.results.map((item) => item.asin), ["B0MATCH001"]);
assert.match(result.filteredOutReasons["B0NOPRIME1"]?.join(" ") ?? "", /Prime delivery not verified/);
assert.match(result.filteredOutReasons["B0NARROW01"]?.join(" ") ?? "", /width 65/);
assert.ok(result.results[0]?.matchedFilters.includes("width >= 77 inches"));
assert.ok(result.results[0]?.matchedFilters.includes("Prime delivery"));
assert.ok(result.results[0]?.matchedFilters.includes("delivery by tomorrow"));
});
it("sorts by price when requested", () => {
const result = applyFiltersAndLimit([
product({ asin: "B0EXPENSIV", rating: 4.9, reviewCount: 1000, price: { amount: 500, currency: "USD", display: "$500.00" } }),
product({ asin: "B0CHEAPER1", rating: 4.1, reviewCount: 300, price: { amount: 200, currency: "USD", display: "$200.00" } })
], { includeKeywords: [], excludeKeywords: [], sortBy: "price" }, 10);
assert.deepEqual(result.results.map((item) => item.asin), ["B0CHEAPER1", "B0EXPENSIV"]);
});
});
+3
View File
@@ -0,0 +1,3 @@
# Fixtures
Fixtures in this directory are hand-crafted sanitized HTML snippets. They are not live Amazon snapshots and contain no cookies, account details, delivery location, scripts, tracking identifiers, or browser profile data.
@@ -0,0 +1,30 @@
<!-- Hand-crafted sanitized fixture. Not a live Amazon snapshot. -->
<html>
<body>
<h1 id="productTitle">Bright Daylight LED Bulbs 100W Equivalent, 50 Count</h1>
<span id="productTitle_feature_div"></span>
<span class="a-price"><span class="a-offscreen">$18.99</span></span>
<div id="mir-layout-DELIVERY_BLOCK-slot-PRIMARY_DELIVERY_MESSAGE_LARGE">FREE delivery Tomorrow</div>
<div id="availability">In Stock</div>
<div id="merchant-info">Ships from Amazon.com</div>
<div id="feature-bullets">
<ul>
<li><span>Energy efficient 100W equivalent bulbs.</span></li>
<li><span>Daylight color temperature for kitchens and garages.</span></li>
</ul>
</div>
<table id="productOverview_feature_div">
<tr><td>Brand</td><td>BrightCo</td></tr>
<tr><td>Light Type</td><td>LED</td></tr>
</table>
<span id="acrPopover" title="4.6 out of 5 stars"></span>
<span id="acrCustomerReviewText">1,234 ratings</span>
<table id="histogramTable">
<tr><td>5 star</td><td>72%</td></tr>
<tr><td>4 star</td><td>15%</td></tr>
<tr><td>3 star</td><td>7%</td></tr>
<tr><td>2 star</td><td>3%</td></tr>
<tr><td>1 star</td><td>3%</td></tr>
</table>
</body>
</html>
@@ -0,0 +1,23 @@
<!-- Hand-crafted sanitized fixture. Not a live Amazon snapshot. -->
<html>
<body>
<div data-component-type="s-search-result" data-asin="B0TESTLED1">
<h2><a class="a-link-normal s-line-clamp-2" href="/Bright-Daylight-Equivalent/dp/B0TESTLED1/ref=sr_1_1">Bright Daylight 100W Equivalent LED Bulbs, 50 Count</a></h2>
<span class="a-price"><span class="a-offscreen">$18.99</span></span>
<span class="a-size-base a-color-secondary">$0.38/Count</span>
<span aria-label="4.6 out of 5 stars"></span>
<a aria-label="1,234 ratings"></a>
<div class="a-row a-size-base a-color-secondary">FREE delivery Tomorrow</div>
<img class="s-image" src="https://m.media-amazon.com/images/I/test-led.jpg" />
</div>
<div data-component-type="s-search-result" data-asin="B0TESTLED2">
<span>Sponsored</span>
<h2><a href="https://www.amazon.com/gp/product/B0TESTLED2">Value LED Bulbs Soft White, Pack of 24</a></h2>
<span class="a-price"><span class="a-offscreen">$21.99</span></span>
<span aria-label="4.3 out of 5 stars"></span>
<a aria-label="543 ratings"></a>
<div>Delivery Friday</div>
</div>
<a class="s-pagination-next" href="/s?k=led+bulbs&amp;page=2">Next</a>
</body>
</html>
@@ -0,0 +1,83 @@
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import {
extractUnitCount,
parseMoney,
parseRating,
parseReviewCount,
parseStarBreakdown,
parseUnitPrice
} from "../src/parsers.js";
describe("parsers", () => {
it("parses USD money", () => {
assert.deepEqual(parseMoney("$19.99"), { amount: 19.99, currency: "USD", display: "$19.99" });
});
it("parses rating text", () => {
assert.equal(parseRating("4.6 out of 5 stars"), 4.6);
});
it("parses review count text", () => {
assert.equal(parseReviewCount("1,234 ratings"), 1234);
});
it("parses visible star histogram percentages", () => {
assert.deepEqual(parseStarBreakdown("5 star 72% 4 star 15% 3 star 7% 2 star 3% 1 star 3%"), {
five: 72,
four: 15,
three: 7,
two: 3,
one: 3,
basis: "percent"
});
});
it("extracts high-confidence unit counts", () => {
assert.deepEqual(extractUnitCount("LED bulbs, 100 Count, daylight"), {
count: 100,
confidence: "high",
source: "100 Count"
});
assert.deepEqual(extractUnitCount("Pack of 6 USB-C cables"), {
count: 6,
confidence: "high",
source: "Pack of 6"
});
});
it("distinguishes lower-confidence unit count phrases", () => {
assert.deepEqual(extractUnitCount("Set of 8 replacement filters"), {
count: 8,
confidence: "medium",
source: "Set of 8"
});
assert.deepEqual(extractUnitCount("6 bulbs soft white"), {
count: 6,
confidence: "low",
source: "6 bulbs"
});
});
it("parses visible unit prices", () => {
assert.deepEqual(parseUnitPrice("$0.33/Count"), {
amount: 0.33,
currency: "USD",
display: "$0.33/Count"
});
});
it("prefers the unit price when product price appears first", () => {
assert.deepEqual(parseUnitPrice("$9.99 ($5.00$5.00/count)"), {
amount: 5,
currency: "USD",
display: "$5.00/count"
});
});
it("parses whole-dollar and one-decimal prices", () => {
assert.deepEqual(parseMoney("$20"), { amount: 20, currency: "USD", display: "$20" });
assert.deepEqual(parseMoney("$19.9"), { amount: 19.9, currency: "USD", display: "$19.9" });
});
});
@@ -0,0 +1,56 @@
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import { extractWidthInches, formatWidthInches } from "../src/product-metrics.js";
import type { ProductSearchResult } from "../src/types.js";
function product(overrides: Partial<ProductSearchResult>): ProductSearchResult {
return {
asin: "B0WIDTH001",
title: "Base Product",
url: "https://www.amazon.com/dp/B0WIDTH001",
specs: [],
bullets: [],
matchedFilters: [],
missingFields: [],
extractionNotes: [],
...overrides
};
}
describe("product metrics", () => {
it("extracts explicit W dimensions from overall product specs", () => {
const width = extractWidthInches(product({
specs: [{ name: "Product Dimensions", value: "35\"D x 83.4\"W x 31\"H" }]
}));
assert.equal(width, 83.4);
});
it("uses dimension order labels when W is not repeated in the value", () => {
const width = extractWidthInches(product({
specs: [{ name: "Item Dimensions D x W x H", value: "35 x 108 x 31 inches" }]
}));
assert.equal(width, 108);
});
it("ignores non-overall width specs before falling back to title width", () => {
const width = extractWidthInches(product({
title: "83 Inch Sofa Bed",
specs: [
{ name: "Seat Interior Width", value: "65 Inches" },
{ name: "Arm Width", value: "5 Inches" },
{ name: "Minimum Required Door Width", value: "72 Inches" }
]
}));
assert.equal(width, 83);
});
it("formats unknown and decimal widths", () => {
assert.equal(formatWidthInches(undefined), "unknown");
assert.equal(formatWidthInches(83.4), "83.4\"");
assert.equal(formatWidthInches(108), "108\"");
});
});
@@ -0,0 +1,84 @@
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import { parseNaturalLanguageRequest } from "../src/query-parser.js";
describe("parseNaturalLanguageRequest", () => {
it("extracts the target LED bulb filters from natural language", () => {
const parsed = parseNaturalLanguageRequest(
"100w led bulbs that cost less than $4 each and have over 200 reviews with a review score of more than 4.5 stars"
);
assert.equal(parsed.query, "100w led bulbs");
assert.equal(parsed.filters.maxUnitPrice, 4);
assert.equal(parsed.filters.minReviews, 200);
assert.equal(parsed.filters.reviewCountComparison, "gt");
assert.equal(parsed.filters.minRating, 4.5);
assert.equal(parsed.filters.ratingComparison, "gt");
});
it("distinguishes inclusive review and rating phrasing", () => {
const parsed = parseNaturalLanguageRequest("usb c charger at least 500 reviews and 4.3 stars or better");
assert.equal(parsed.query, "usb c charger");
assert.equal(parsed.filters.minReviews, 500);
assert.equal(parsed.filters.reviewCountComparison, "gte");
assert.equal(parsed.filters.minRating, 4.3);
assert.equal(parsed.filters.ratingComparison, "gte");
});
it("cleans rating filter phrases from search query text", () => {
const parsed = parseNaturalLanguageRequest("usb c cable with over 1000 reviews and rating over 4 stars");
assert.equal(parsed.query, "usb c cable");
assert.equal(parsed.filters.minReviews, 1000);
assert.equal(parsed.filters.minRating, 4);
});
it("extracts rating filters without requiring the word of", () => {
const parsed = parseNaturalLanguageRequest(
"sofa bed, 77 inches or wider, over 50 reviews, rating 4.0 or better, Prime delivery only, sort by price low to high"
);
assert.equal(parsed.query, "sofa bed");
assert.equal(parsed.filters.minWidthInches, 77);
assert.equal(parsed.filters.widthComparison, "gte");
assert.equal(parsed.filters.minReviews, 50);
assert.equal(parsed.filters.reviewCountComparison, "gt");
assert.equal(parsed.filters.minRating, 4);
assert.equal(parsed.filters.ratingComparison, "gte");
assert.equal(parsed.filters.requirePrime, true);
assert.equal(parsed.filters.sortBy, "price");
});
it("extracts limit and max product price phrases", () => {
const parsed = parseNaturalLanguageRequest("return 5 wireless mouse under $30");
assert.equal(parsed.query, "wireless mouse");
assert.equal(parsed.limit, 5);
assert.equal(parsed.filters.maxPrice, 30);
});
it("extracts sofa width, Prime, and delivery urgency filters", () => {
const parsed = parseNaturalLanguageRequest(
"sofa bed of 77inches or wider in width, review score of 4 stars and higher, 200+ reviews and shipped with prime, color beige if possible, delivery by tomorrow"
);
assert.equal(parsed.query, "sofa bed color beige if possible");
assert.equal(parsed.filters.minWidthInches, 77);
assert.equal(parsed.filters.minRating, 4);
assert.equal(parsed.filters.ratingComparison, "gte");
assert.equal(parsed.filters.minReviews, 200);
assert.equal(parsed.filters.reviewCountComparison, "gte");
assert.equal(parsed.filters.requirePrime, true);
assert.equal(parsed.filters.deliveryBy, "tomorrow");
});
it("extracts overnight delivery requests", () => {
const parsed = parseNaturalLanguageRequest("queen sleeper sofa with overnight shipping and Prime");
assert.equal(parsed.query, "queen sleeper sofa");
assert.equal(parsed.filters.requirePrime, true);
assert.equal(parsed.filters.deliveryBy, "overnight");
});
});
@@ -0,0 +1,99 @@
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import { createMarkdownReport, createResponse } from "../src/report.js";
describe("report", () => {
it("creates a structured JSON response", () => {
const response = createResponse({
query: "usb c cable",
filters: { includeKeywords: [], excludeKeywords: [], minReviews: 1000 },
limit: 1,
maxSearchPages: 2,
results: [],
filteredOutCount: 4,
warnings: ["partial extraction"],
now: () => new Date("2026-04-15T00:00:00.000Z")
});
assert.equal(response.source.site, "amazon.com");
assert.equal(response.filteredOutCount, 4);
assert.equal(response.source.scrapedAt, "2026-04-15T00:00:00.000Z");
});
it("creates concise markdown with product details and warnings", () => {
const markdown = createMarkdownReport(createResponse({
query: "usb c cable",
filters: { includeKeywords: [], excludeKeywords: [] },
limit: 1,
maxSearchPages: 2,
filteredOutCount: 0,
warnings: ["price missing for one item"],
now: () => new Date("2026-04-15T00:00:00.000Z"),
results: [{
asin: "B0TEST0001",
title: "USB-C Cable",
url: "https://www.amazon.com/dp/B0TEST0001",
price: { amount: 9.99, currency: "USD", display: "$9.99" },
rating: 4.7,
reviewCount: 1234,
delivery: { display: "FREE delivery Tomorrow", free: true },
specs: [{ name: "Length", value: "6 ft" }],
bullets: ["Braided cable"],
matchedFilters: [],
missingFields: ["starBreakdown"],
extractionNotes: []
}]
}));
assert.match(markdown, /USB-C Cable/);
assert.match(markdown, /\$9\.99/);
assert.match(markdown, /4\.7 stars/);
assert.match(markdown, /price missing/);
assert.match(markdown, /https:\/\/www\.amazon\.com\/dp\/B0TEST0001/);
});
it("creates a chat-safe template with direct product links and constraint status markers", () => {
const markdown = createMarkdownReport(createResponse({
query: "sofa bed beige",
filters: {
includeKeywords: [],
excludeKeywords: [],
minRating: 4,
minReviews: 200,
minWidthInches: 77,
requirePrime: true,
deliveryBy: "tomorrow"
},
limit: 10,
maxSearchPages: 2,
filteredOutCount: 3,
warnings: [],
now: () => new Date("2026-04-15T00:00:00.000Z"),
results: [{
asin: "B0SOFABED1",
title: "HONBAY Modular Sectional Sleeper",
url: "https://www.amazon.com/dp/B0SOFABED1",
price: { amount: 539.99, currency: "USD", display: "$539.99" },
rating: 4.1,
reviewCount: 242,
delivery: { display: "FREE delivery Tomorrow", free: true, prime: true },
specs: [{ name: "Item Dimensions D x W x H", value: "83.4\"D x 83.4\"W x 35\"H" }],
bullets: [],
matchedFilters: [],
missingFields: ["starBreakdown"],
extractionNotes: []
}]
}));
assert.match(markdown, /## Best Matches/);
assert.doesNotMatch(markdown, /\| # \| Product \| Price \| Rating \| Reviews \| Width \| Prime \| Delivery \| Link \|/);
assert.doesNotMatch(markdown, /\[Amazon\]\(https:\/\/www\.amazon\.com\/dp\/B0SOFABED1\)/);
assert.match(markdown, /HONBAY Modular Sectional Sleeper/);
assert.match(markdown, /Link: https:\/\/www\.amazon\.com\/dp\/B0SOFABED1/);
assert.match(markdown, /83\.4" OK/);
assert.match(markdown, /Prime OK/);
assert.match(markdown, /Tomorrow OK/);
assert.match(markdown, /Price: \$539\.99/);
});
});
@@ -0,0 +1,95 @@
import assert from "node:assert/strict";
import { readFile } from "node:fs/promises";
import { join } from "node:path";
import { describe, it } from "node:test";
import { extractSearchPage } from "../src/search-page.js";
const fixturePath = join(import.meta.dirname, "fixtures", "search-results.html");
describe("extractSearchPage", () => {
it("extracts normalized product candidates from sanitized search HTML", async () => {
const html = await readFile(fixturePath, "utf8");
const extracted = extractSearchPage(html, "https://www.amazon.com/s?k=led+bulbs");
assert.equal(extracted.status, "ok");
assert.equal(extracted.products.length, 2);
assert.equal(extracted.products[0]?.asin, "B0TESTLED1");
assert.equal(extracted.products[0]?.url, "https://www.amazon.com/dp/B0TESTLED1");
assert.equal(extracted.products[0]?.price?.amount, 18.99);
assert.equal(extracted.products[0]?.unitPrice?.amount, 0.38);
assert.equal(extracted.products[0]?.rating, 4.6);
assert.equal(extracted.products[0]?.reviewCount, 1234);
assert.equal(extracted.products[0]?.delivery?.free, true);
assert.equal(extracted.products[0]?.isSponsored, false);
assert.equal(extracted.products[1]?.isSponsored, true);
assert.equal(extracted.nextPageUrl, "https://www.amazon.com/s?k=led+bulbs&page=2");
});
it("detects Amazon challenge pages", () => {
const extracted = extractSearchPage("<html><title>Robot Check</title><body>Enter the characters you see below</body></html>", "https://www.amazon.com/s?k=x");
assert.equal(extracted.status, "challenge");
assert.match(extracted.warnings[0] ?? "", /challenge/i);
assert.equal(extracted.products.length, 0);
});
it("returns ok with no products for empty or cardless pages", () => {
const extracted = extractSearchPage("<html><body>No results</body></html>", "https://www.amazon.com/s?k=x");
assert.equal(extracted.status, "ok");
assert.deepEqual(extracted.products, []);
assert.equal(extracted.nextPageUrl, undefined);
});
it("skips malformed ASINs and cards without titles", () => {
const extracted = extractSearchPage(`
<div data-asin="bad"><h2><a href="/dp/bad">Bad ASIN</a></h2></div>
<div data-asin="B0VALID1234"></div>
`, "https://www.amazon.com/s?k=x");
assert.equal(extracted.status, "ok");
assert.equal(extracted.products.length, 0);
});
it("keeps candidates with missing price and records missing price later", () => {
const extracted = extractSearchPage(`
<div data-asin="B0NOPRICE1">
<h2><a href="/dp/B0NOPRICE1">No Price Product</a></h2>
</div>
`, "https://www.amazon.com/s?k=x");
assert.equal(extracted.products.length, 1);
assert.equal(extracted.products[0]?.price, undefined);
});
it("detects Prime badges even when visible delivery text omits the word Prime", () => {
const extracted = extractSearchPage(`
<div data-asin="B0PRIME123">
<h2><a href="/dp/B0PRIME123">Prime Sofa Bed</a></h2>
<span class="a-price"><span class="a-offscreen">$299.99</span></span>
<span aria-label="4.4 out of 5 stars"></span>
<span aria-label="246 ratings"></span>
<i class="a-icon a-icon-prime" aria-label="Amazon Prime"></i>
<span>FREE delivery Tomorrow</span>
</div>
`, "https://www.amazon.com/s?k=sofa+bed");
assert.equal(extracted.products.length, 1);
assert.equal(extracted.products[0]?.delivery?.prime, true);
assert.equal(extracted.products[0]?.delivery?.free, true);
assert.match(extracted.products[0]?.delivery?.display ?? "", /Tomorrow/);
});
it("does not treat Prime in a product title as Prime delivery", () => {
const extracted = extractSearchPage(`
<div data-asin="B0TITLE123">
<h2><a href="/dp/B0TITLE123">Prime Sofa Bed</a></h2>
<span>FREE delivery Tomorrow</span>
</div>
`, "https://www.amazon.com/s?k=sofa+bed");
assert.equal(extracted.products.length, 1);
assert.equal(extracted.products[0]?.delivery?.prime, false);
});
});
@@ -0,0 +1,46 @@
import assert from "node:assert/strict";
import { mkdtemp, mkdir, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { describe, it } from "node:test";
import { resolveWebAutomationRuntime } from "../src/web-automation-runtime.js";
async function createRuntime() {
const dir = await mkdtemp(join(tmpdir(), "amazon-shopping-runtime-"));
await writeFile(join(dir, "check-install.js"), "console.log('ok');\n");
await writeFile(join(dir, "package.json"), "{\"type\":\"module\"}\n");
await mkdir(join(dir, "node_modules", ".bin"), { recursive: true });
await writeFile(join(dir, "node_modules", ".bin", "tsx"), "#!/usr/bin/env node\n");
return dir;
}
describe("resolveWebAutomationRuntime", () => {
it("uses AMAZON_SHOPPING_WEB_AUTOMATION_DIR first", async () => {
const runtimeDir = await createRuntime();
const resolved = await resolveWebAutomationRuntime({
env: { AMAZON_SHOPPING_WEB_AUTOMATION_DIR: runtimeDir },
homeDir: "/missing-home",
skillDir: "/missing-skill"
});
assert.equal(resolved.scriptsDir, runtimeDir);
assert.deepEqual(resolved.checkInstall, {
cwd: runtimeDir,
command: "node",
args: ["check-install.js"]
});
});
it("returns a clear error when required files are missing", async () => {
const dir = await mkdtemp(join(tmpdir(), "amazon-shopping-runtime-missing-"));
await assert.rejects(
() => resolveWebAutomationRuntime({
env: { AMAZON_SHOPPING_WEB_AUTOMATION_DIR: dir },
homeDir: "/missing-home",
skillDir: "/missing-skill"
}),
/check-install.js/
);
});
});
+13
View File
@@ -0,0 +1,13 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"strict": true,
"esModuleInterop": true,
"forceConsistentCasingInFileNames": true,
"skipLibCheck": true,
"types": ["node"]
},
"include": ["src/**/*.ts", "tests/**/*.ts"]
}
-46
View File
@@ -1,46 +0,0 @@
---
name: elevenlabs-stt
description: Transcribe audio files with ElevenLabs Speech-to-Text (Scribe v2) from the local CLI. Use when you need local audio transcription with optional speaker diarization, language hints, event tagging, or JSON output via scripts/transcribe.sh.
---
# ElevenLabs Speech-to-Text
Use `scripts/transcribe.sh` to transcribe a local audio file with ElevenLabs STT.
## Requirements
Preferred: set `ELEVENLABS_API_KEY` in the environment before running the script.
Fallback: if the environment variable is not set, the script will try to read the key from local OpenClaw config files in `~/.openclaw/`.
Required binaries:
- `curl`
- `jq`
- `python3`
## Usage
Run from the skill directory or call the script by full path.
Examples:
```bash
scripts/transcribe.sh /path/to/audio.mp3
scripts/transcribe.sh /path/to/audio.mp3 --diarize --lang en
scripts/transcribe.sh /path/to/audio.mp3 --json
scripts/transcribe.sh /path/to/audio.mp3 --events
```
## Options
- `--diarize` — enable speaker diarization
- `--lang CODE` — pass an ISO language code hint such as `en`, `es`, or `fr`
- `--json` — print the full JSON response instead of only transcript text
- `--events` — include audio event tagging when supported
## Notes
- The script uploads a local file directly to ElevenLabs.
- The model is fixed to `scribe_v2` in the current script.
- The script returns plain transcript text by default, or pretty-printed JSON with `--json`.
- If the API returns an error payload, the script prints the error and exits non-zero.
-143
View File
@@ -1,143 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
# ElevenLabs Speech-to-Text transcription script
# Usage: transcribe.sh <audio_file> [options]
show_help() {
cat << EOF
Usage: $(basename "$0") <audio_file> [options]
Options:
--diarize Enable speaker diarization
--lang CODE ISO language code (e.g., en, pt, es, fr)
--json Output full JSON response
--events Tag audio events (laughter, music, etc.)
-h, --help Show this help
Environment:
ELEVENLABS_API_KEY Required API key
Examples:
$(basename "$0") voice_note.ogg
$(basename "$0") meeting.mp3 --diarize --lang en
$(basename "$0") podcast.mp3 --json > transcript.json
EOF
exit 0
}
# Defaults
DIARIZE="false"
LANG_CODE=""
JSON_OUTPUT="false"
TAG_EVENTS="false"
FILE=""
# Parse arguments
while [[ $# -gt 0 ]]; do
case $1 in
-h|--help) show_help ;;
--diarize) DIARIZE="true"; shift ;;
--lang) LANG_CODE="$2"; shift 2 ;;
--json) JSON_OUTPUT="true"; shift ;;
--events) TAG_EVENTS="true"; shift ;;
-*) echo "Unknown option: $1" >&2; exit 1 ;;
*) FILE="$1"; shift ;;
esac
done
# Validate
if [[ -z "$FILE" ]]; then
echo "Error: No audio file specified" >&2
show_help
fi
if [[ ! -f "$FILE" ]]; then
echo "Error: File not found: $FILE" >&2
exit 1
fi
# API key (check env, then fall back to local OpenClaw config/secrets)
API_KEY="${ELEVENLABS_API_KEY:-}"
if [[ -z "$API_KEY" ]]; then
OPENCLAW_DIR="${HOME}/.openclaw"
for CANDIDATE in "$OPENCLAW_DIR/secrets.json" "$OPENCLAW_DIR/openclaw.json"; do
if [[ -f "$CANDIDATE" ]]; then
API_KEY=$(python3 - "$CANDIDATE" <<'PY'
import json, sys
path = sys.argv[1]
try:
with open(path) as f:
data = json.load(f)
except Exception:
print("")
raise SystemExit(0)
candidates = [
("elevenlabs", "apiKey"),
("messages", "tts", "elevenlabs", "apiKey"),
]
for cand in candidates:
cur = data
ok = True
for key in cand:
if isinstance(cur, dict) and key in cur:
cur = cur[key]
else:
ok = False
break
if ok and isinstance(cur, str) and cur:
print(cur)
raise SystemExit(0)
print("")
PY
)
if [[ -n "$API_KEY" ]]; then
break
fi
fi
done
fi
if [[ -z "$API_KEY" ]]; then
echo "Error: ELEVENLABS_API_KEY not set and no local OpenClaw ElevenLabs key was found" >&2
exit 1
fi
# Build curl command
CURL_ARGS=(
-s
-X POST
"https://api.elevenlabs.io/v1/speech-to-text"
-H "xi-api-key: $API_KEY"
-F "file=@$FILE"
-F "model_id=scribe_v2"
-F "diarize=$DIARIZE"
-F "tag_audio_events=$TAG_EVENTS"
)
if [[ -n "$LANG_CODE" ]]; then
CURL_ARGS+=(-F "language_code=$LANG_CODE")
fi
# Make request
RESPONSE=$(curl "${CURL_ARGS[@]}")
# Check for errors
if echo "$RESPONSE" | grep -q '"detail"'; then
echo "Error from API:" >&2
echo "$RESPONSE" | jq -r '.detail.message // .detail' >&2
exit 1
fi
# Output
if [[ "$JSON_OUTPUT" == "true" ]]; then
echo "$RESPONSE" | jq .
else
# Extract just the text
TEXT=$(echo "$RESPONSE" | jq -r '.text // empty')
if [[ -n "$TEXT" ]]; then
echo "$TEXT"
else
echo "$RESPONSE"
fi
fi
+159
View File
@@ -0,0 +1,159 @@
---
name: flight-finder
description: Use when a user wants a flight-search report with explicit trip constraints, ranked options, and PDF/email delivery. Prefer when the request may include split returns, flexible dates, passenger groups, market-localized pricing, or a WhatsApp-safe workflow that should keep going after status nudges instead of hanging behind a long inline prompt.
---
# Flight Finder
Use this skill instead of the old `docs/prompts/dfw-blq-2026.md` prompt when the user wants a flight-search report.
The deliverable target is always a PDF report plus email delivery when the report is complete.
PDF output rule:
- Render aggregator booking links for every quoted fare.
- Render direct-airline booking links when they were actually captured.
- If a direct-airline link was not captured for a quote, say so explicitly in the report instead of implying one exists.
Freshness rule:
- Never reuse cached flight-search captures, prior same-day report payloads, earlier workspace artifacts, or previously rendered PDFs as the primary evidence for a new run.
- Every flight-finder run must execute a fresh bounded search before ranking, PDF render, or email delivery.
- If a fresh search cannot be completed, say so clearly and stop with a degraded/incomplete outcome instead of silently reusing old captures.
## Inputs
Accept either:
- a natural-language trip request
- a structured helper payload
- a route plus partial constraints that still need follow-up questions
Collect these explicitly when missing:
- origin and destination
- outbound date window
- whether there are one or more return legs
- passenger groups
- which passenger groups travel on which legs
- flexibility rules
- cabin / stop / layover preferences when relevant
- exclusions such as airlines, countries, or airports when relevant
- recipient email before final render/send
Optional explicit input:
- `marketCountry`
- if present, it must be a validated ISO 3166-1 alpha-2 uppercase country code such as `TH` or `DE`
- if present, it activates the VPN / market-localized search path for the bounded search phase only
- if absent, do not change VPN in this implementation pass
Do not silently infer or reuse missing trip-shape details from earlier runs just because the route looks similar.
If the user already explicitly said where to email the finished PDF, that counts as delivery authorization once the report is complete. Do not ask for a second send confirmation unless the destination changed or the user sounded uncertain.
## Core workflow
1. Normalize the trip request into typed legs, passenger groups, assignments, and preferences.
2. Ask targeted follow-up questions for missing search-critical inputs.
3. Run the bounded search phase across the configured travel sources.
4. Rank the viable combinations in USD.
5. Assemble the final report payload.
6. Render the PDF.
7. Send the email from Luke to the user-specified recipient.
Completion rule:
- A partial source result is not completion.
- A partial helper result is not completion.
- The workflow is complete only when the final report payload exists, the PDF is rendered, the email was sent, and any VPN cleanup required by the run has been attempted.
- An interim commentary/status line is not completion.
- For isolated cron-style runs, do not leave the run with only a phase-update summary. If PDF render or email send did not happen, the terminal result must say so explicitly as a failed or incomplete outcome.
## WhatsApp-safe behavior
Follow the same operational style as `property-assessor`:
- missing required input should trigger a direct question, not an invented assumption
- `update?`, `status?`, `and?`, and similar nudges mean “report status and keep going”
- a silent helper/source path is a failed path and should be abandoned quickly
- previously saved workspace captures are not a fallback; they are stale evidence and must not be reused for a new report
- keep progress state by phase so a dropped session can resume or at least report the last completed phase
- do not start email-tool exploration before the report payload is complete and the PDF is ready to render
- if PDF render fails, return the completed report summary in chat and report delivery failure separately
- if email send fails after render, say so clearly and keep the rendered PDF path
## Long-running command rule
- Do not fire a long-running local command and then abandon it.
- If an `exec` call returns a running session/process handle instead of a completed result, immediately follow it with the relevant process polling/log calls until the command finishes, fails, or times out.
- Treat VPN connect, fresh search sweeps, PDF render, and Gmail send as critical-path commands. They must be observed to completion; do not assume they succeeded just because the process started.
- If a critical-path command cannot be observed to completion, stop claiming progress and report the run as incomplete.
## Isolated cron rule
- In isolated cron runs, success means all of the following are true:
- fresh search artifacts exist for the current run
- final `report-payload.json` exists for the current run
- PDF render completed
- Gmail send completed to the requested recipient
- If the model/provider times out, the VPN step flakes, or the run ends before those artifacts exist, report a failure/incomplete outcome rather than an `ok`-sounding completion summary.
- Do not end an isolated cron run on the VPN/connect phase alone.
## Search-source contract
Use the proven travel-source order from the local viability findings:
1. KAYAK
2. Skyscanner
3. Expedia
4. airline direct-booking cross-check
Rules:
- treat source viability as evidence-based, not assumed
- if a source is blocked or degraded, continue with the remaining sources and record the issue in the report
- keep the bounded search phase observable with concise updates
- normalize compared prices to USD before ranking
- do not claim market-localized pricing succeeded unless VPN connect and post-connect verification succeeded
- do not inspect prior `workspace/reports` flight artifacts as reusable search input for a new run
## VPN / market-country rules
- Use VPN only when `marketCountry` is explicitly provided in the typed request for this implementation pass.
- Connect VPN only for the bounded search phase.
- Bounded search phase means: from first travel-site navigation until the last search-result capture for the report.
- If VPN connect or verification fails within the configured timeout, continue on the default market and mark the report as degraded.
- Disconnect VPN immediately after the bounded search phase, before ranking/render/delivery.
- On rollback or failure, attempt disconnect immediately.
## Delivery rules
- Sender must be Luke: `luke@fiorinis.com`
- Delivery path must be Lukes wrapper:
- `zsh ~/.openclaw/workspace/bin/gog-luke gmail send --to "<target>" --subject "<subject>" --body "<body>" --attach "<pdf-path>"`
- If recipient email is missing, ask for it before the render/send phase.
- If the user already provided the recipient email, do not ask for a redundant “send it” confirmation.
## Helper commands
From `~/.openclaw/workspace/skills/flight-finder/` or the repo mirror copy:
```bash
npm run normalize-request -- --legacy-dfw-blq
npm run normalize-request -- --input "<request.json>"
npm run report-status -- --input "<report-payload.json>"
npm run render-report -- --input "<report-payload.json>" --output "<report.pdf>"
npm run delivery-plan -- --to "<recipient@example.com>" --subject "<subject>" --body "<body>" --attach "<report.pdf>"
```
Rules:
- `normalize-request` should report missing search inputs separately from delivery-only email gaps
- `report-status` should expose whether the run is ready to search, ready for a chat summary, ready to render a PDF, or ready to email
- `render-report` must reject incomplete report payloads
- `render-report` must reject payloads that are not explicitly marked as the output of a fresh search run
- `delivery-plan` must stay on the Luke sender path and must not silently fall back to another sender
## Prompt migration rule
The old DFW↔BLQ prompt is legacy input only. Do not treat it as freeform execution prose. Its file should contain a literal `use flight finder skill with the following json payload` request so manual runs and scheduled runs stay aligned.
@@ -0,0 +1,168 @@
{
"request": {
"tripName": "DFW ↔ BLQ flight report",
"legs": [
{
"id": "outbound",
"origin": "DFW",
"destination": "BLQ",
"earliest": "2026-05-30",
"latest": "2026-06-07",
"label": "Outbound"
},
{
"id": "return-pair",
"origin": "BLQ",
"destination": "DFW",
"relativeToLegId": "outbound",
"minDaysAfter": 6,
"maxDaysAfter": 10,
"label": "Return for 2 adults"
},
{
"id": "return-solo",
"origin": "BLQ",
"destination": "DFW",
"earliest": "2026-06-28",
"latest": "2026-07-05",
"label": "Return for 1 adult"
}
],
"passengerGroups": [
{
"id": "pair",
"adults": 2,
"label": "2 adults traveling together"
},
{
"id": "solo",
"adults": 1,
"label": "1 adult returning separately"
}
],
"legAssignments": [
{
"legId": "outbound",
"passengerGroupIds": ["pair", "solo"]
},
{
"legId": "return-pair",
"passengerGroupIds": ["pair"]
},
{
"legId": "return-solo",
"passengerGroupIds": ["solo"]
}
],
"recipientEmail": "stefano@fiorinis.com",
"preferences": {
"preferOneStop": true,
"maxStops": 1,
"maxLayoverHours": 6,
"flexibleDates": true,
"excludeAirlines": ["Turkish Airlines"],
"excludeCountries": ["TR"],
"requireAirlineDirectCrossCheck": true,
"specialConstraints": [
"Exclude itineraries operated by airlines based in Arab or Middle Eastern countries.",
"Exclude itineraries routing through airports in Arab or Middle Eastern countries.",
"Start from a fresh browser/session profile for this search."
],
"geoPricingMarket": "Thailand",
"marketCountry": "TH",
"normalizeCurrencyTo": "USD"
}
},
"searchExecution": {
"freshSearch": true,
"startedAt": "2026-03-30T21:00:00Z",
"completedAt": "2026-03-30T21:20:00Z",
"artifactsRoot": "/Users/stefano/.openclaw/workspace/reports/dfw-blq-flight-report-2026-03-30-exec",
"notes": [
"Fresh bounded Thailand-market search executed for this report payload.",
"Prior workspace captures must not be reused across flight-finder runs."
]
},
"sourceFindings": [
{
"source": "kayak",
"status": "viable",
"checkedAt": "2026-03-30T21:00:00Z",
"notes": [
"KAYAK returned multiple one-stop DFW -> BLQ options and exposed direct-booking hints for British Airways."
]
},
{
"source": "skyscanner",
"status": "viable",
"checkedAt": "2026-03-30T21:05:00Z",
"notes": [
"Skyscanner returned one-stop and multi-stop DFW -> BLQ results with total USD pricing."
]
},
{
"source": "expedia",
"status": "viable",
"checkedAt": "2026-03-30T21:10:00Z",
"notes": [
"Expedia returned one-way DFW -> BLQ options with clear per-traveler pricing."
]
},
{
"source": "airline-direct",
"status": "degraded",
"checkedAt": "2026-03-30T21:15:00Z",
"notes": [
"United's direct booking shell loaded with the route/date context but failed to complete the search."
]
}
],
"quotes": [
{
"id": "kayak-outbound-ba",
"source": "kayak",
"legId": "outbound",
"passengerGroupIds": ["pair", "solo"],
"bookingLink": "https://www.kayak.com/flights/DFW-BLQ/2026-05-30?adults=3&sort=bestflight_a&fs=stops=-2",
"itinerarySummary": "British Airways via London Heathrow",
"airlineName": "British Airways",
"departureTimeLocal": "10:59 PM",
"arrivalTimeLocal": "11:45 PM +1",
"stopsText": "1 stop",
"layoverText": "6h 15m in London Heathrow",
"totalDurationText": "17h 46m",
"totalPriceUsd": 2631,
"displayPriceUsd": "$2,631 total",
"directBookingUrl": "https://www.britishairways.com/",
"crossCheckStatus": "failed",
"notes": [
"Observed on KAYAK at $877 per traveler for 3 adults.",
"Airline-direct cross-check remains degraded in this implementation pass."
]
}
],
"rankedOptions": [
{
"id": "primary-outbound",
"title": "Best observed outbound baseline",
"quoteIds": ["kayak-outbound-ba"],
"totalPriceUsd": 2631,
"rationale": "Lowest observed one-stop fare in the bounded spike while still meeting the layover constraint."
}
],
"executiveSummary": [
"The bounded implementation-pass smoke test produced a viable report payload from real DFW -> BLQ search evidence.",
"KAYAK, Skyscanner, and Expedia all returned usable results on this machine.",
"Direct-airline cross-checking is currently best-effort and should be reported honestly when it fails."
],
"reportWarnings": [
"This smoke-test payload captures the bounded implementation-pass workflow, not the full old prompt's multi-leg optimization."
],
"degradedReasons": [
"Airline direct-booking cross-check is still degraded and must be treated as best-effort in this pass."
],
"comparisonCurrency": "USD",
"marketCountryUsed": "TH",
"lastCompletedPhase": "ranking",
"generatedAt": "2026-03-30T21:20:00Z"
}
@@ -0,0 +1,74 @@
{
"tripName": "DFW ↔ BLQ flight report",
"legs": [
{
"id": "outbound",
"origin": "DFW",
"destination": "BLQ",
"earliest": "2026-05-30",
"latest": "2026-06-07",
"label": "Outbound"
},
{
"id": "return-pair",
"origin": "BLQ",
"destination": "DFW",
"relativeToLegId": "outbound",
"minDaysAfter": 6,
"maxDaysAfter": 10,
"label": "Return for 2 adults"
},
{
"id": "return-solo",
"origin": "BLQ",
"destination": "DFW",
"earliest": "2026-06-28",
"latest": "2026-07-05",
"label": "Return for 1 adult"
}
],
"passengerGroups": [
{
"id": "pair",
"adults": 2,
"label": "2 adults traveling together"
},
{
"id": "solo",
"adults": 1,
"label": "1 adult returning separately"
}
],
"legAssignments": [
{
"legId": "outbound",
"passengerGroupIds": ["pair", "solo"]
},
{
"legId": "return-pair",
"passengerGroupIds": ["pair"]
},
{
"legId": "return-solo",
"passengerGroupIds": ["solo"]
}
],
"recipientEmail": "stefano@fiorinis.com",
"preferences": {
"preferOneStop": true,
"maxStops": 1,
"maxLayoverHours": 6,
"flexibleDates": true,
"excludeAirlines": ["Turkish Airlines"],
"excludeCountries": ["TR"],
"requireAirlineDirectCrossCheck": true,
"specialConstraints": [
"Exclude itineraries operated by airlines based in Arab or Middle Eastern countries.",
"Exclude itineraries routing through airports in Arab or Middle Eastern countries.",
"Start from a fresh browser/session profile for this search."
],
"geoPricingMarket": "Thailand",
"marketCountry": "TH",
"normalizeCurrencyTo": "USD"
}
}
+791
View File
@@ -0,0 +1,791 @@
{
"name": "flight-finder-scripts",
"version": "1.0.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "flight-finder-scripts",
"version": "1.0.0",
"dependencies": {
"minimist": "^1.2.8",
"pdfkit": "^0.17.2"
},
"devDependencies": {
"@types/minimist": "^1.2.5",
"@types/pdfkit": "^0.17.3",
"tsx": "^4.20.6",
"typescript": "^5.9.2"
}
},
"node_modules/@esbuild/aix-ppc64": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.27.4.tgz",
"integrity": "sha512-cQPwL2mp2nSmHHJlCyoXgHGhbEPMrEEU5xhkcy3Hs/O7nGZqEpZ2sUtLaL9MORLtDfRvVl2/3PAuEkYZH0Ty8Q==",
"cpu": [
"ppc64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"aix"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-arm": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.27.4.tgz",
"integrity": "sha512-X9bUgvxiC8CHAGKYufLIHGXPJWnr0OCdR0anD2e21vdvgCI8lIfqFbnoeOz7lBjdrAGUhqLZLcQo6MLhTO2DKQ==",
"cpu": [
"arm"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-arm64": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.27.4.tgz",
"integrity": "sha512-gdLscB7v75wRfu7QSm/zg6Rx29VLdy9eTr2t44sfTW7CxwAtQghZ4ZnqHk3/ogz7xao0QAgrkradbBzcqFPasw==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-x64": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.27.4.tgz",
"integrity": "sha512-PzPFnBNVF292sfpfhiyiXCGSn9HZg5BcAz+ivBuSsl6Rk4ga1oEXAamhOXRFyMcjwr2DVtm40G65N3GLeH1Lvw==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/darwin-arm64": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.27.4.tgz",
"integrity": "sha512-b7xaGIwdJlht8ZFCvMkpDN6uiSmnxxK56N2GDTMYPr2/gzvfdQN8rTfBsvVKmIVY/X7EM+/hJKEIbbHs9oA4tQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/darwin-x64": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.27.4.tgz",
"integrity": "sha512-sR+OiKLwd15nmCdqpXMnuJ9W2kpy0KigzqScqHI3Hqwr7IXxBp3Yva+yJwoqh7rE8V77tdoheRYataNKL4QrPw==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/freebsd-arm64": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.27.4.tgz",
"integrity": "sha512-jnfpKe+p79tCnm4GVav68A7tUFeKQwQyLgESwEAUzyxk/TJr4QdGog9sqWNcUbr/bZt/O/HXouspuQDd9JxFSw==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/freebsd-x64": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.27.4.tgz",
"integrity": "sha512-2kb4ceA/CpfUrIcTUl1wrP/9ad9Atrp5J94Lq69w7UwOMolPIGrfLSvAKJp0RTvkPPyn6CIWrNy13kyLikZRZQ==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-arm": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.27.4.tgz",
"integrity": "sha512-aBYgcIxX/wd5n2ys0yESGeYMGF+pv6g0DhZr3G1ZG4jMfruU9Tl1i2Z+Wnj9/KjGz1lTLCcorqE2viePZqj4Eg==",
"cpu": [
"arm"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-arm64": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.27.4.tgz",
"integrity": "sha512-7nQOttdzVGth1iz57kxg9uCz57dxQLHWxopL6mYuYthohPKEK0vU0C3O21CcBK6KDlkYVcnDXY099HcCDXd9dA==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-ia32": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.27.4.tgz",
"integrity": "sha512-oPtixtAIzgvzYcKBQM/qZ3R+9TEUd1aNJQu0HhGyqtx6oS7qTpvjheIWBbes4+qu1bNlo2V4cbkISr8q6gRBFA==",
"cpu": [
"ia32"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-loong64": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.27.4.tgz",
"integrity": "sha512-8mL/vh8qeCoRcFH2nM8wm5uJP+ZcVYGGayMavi8GmRJjuI3g1v6Z7Ni0JJKAJW+m0EtUuARb6Lmp4hMjzCBWzA==",
"cpu": [
"loong64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-mips64el": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.27.4.tgz",
"integrity": "sha512-1RdrWFFiiLIW7LQq9Q2NES+HiD4NyT8Itj9AUeCl0IVCA459WnPhREKgwrpaIfTOe+/2rdntisegiPWn/r/aAw==",
"cpu": [
"mips64el"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-ppc64": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.27.4.tgz",
"integrity": "sha512-tLCwNG47l3sd9lpfyx9LAGEGItCUeRCWeAx6x2Jmbav65nAwoPXfewtAdtbtit/pJFLUWOhpv0FpS6GQAmPrHA==",
"cpu": [
"ppc64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-riscv64": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.27.4.tgz",
"integrity": "sha512-BnASypppbUWyqjd1KIpU4AUBiIhVr6YlHx/cnPgqEkNoVOhHg+YiSVxM1RLfiy4t9cAulbRGTNCKOcqHrEQLIw==",
"cpu": [
"riscv64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-s390x": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.27.4.tgz",
"integrity": "sha512-+eUqgb/Z7vxVLezG8bVB9SfBie89gMueS+I0xYh2tJdw3vqA/0ImZJ2ROeWwVJN59ihBeZ7Tu92dF/5dy5FttA==",
"cpu": [
"s390x"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-x64": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.27.4.tgz",
"integrity": "sha512-S5qOXrKV8BQEzJPVxAwnryi2+Iq5pB40gTEIT69BQONqR7JH1EPIcQ/Uiv9mCnn05jff9umq/5nqzxlqTOg9NA==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/netbsd-arm64": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.27.4.tgz",
"integrity": "sha512-xHT8X4sb0GS8qTqiwzHqpY00C95DPAq7nAwX35Ie/s+LO9830hrMd3oX0ZMKLvy7vsonee73x0lmcdOVXFzd6Q==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/netbsd-x64": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.27.4.tgz",
"integrity": "sha512-RugOvOdXfdyi5Tyv40kgQnI0byv66BFgAqjdgtAKqHoZTbTF2QqfQrFwa7cHEORJf6X2ht+l9ABLMP0dnKYsgg==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openbsd-arm64": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.27.4.tgz",
"integrity": "sha512-2MyL3IAaTX+1/qP0O1SwskwcwCoOI4kV2IBX1xYnDDqthmq5ArrW94qSIKCAuRraMgPOmG0RDTA74mzYNQA9ow==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openbsd-x64": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.27.4.tgz",
"integrity": "sha512-u8fg/jQ5aQDfsnIV6+KwLOf1CmJnfu1ShpwqdwC0uA7ZPwFws55Ngc12vBdeUdnuWoQYx/SOQLGDcdlfXhYmXQ==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openharmony-arm64": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.27.4.tgz",
"integrity": "sha512-JkTZrl6VbyO8lDQO3yv26nNr2RM2yZzNrNHEsj9bm6dOwwu9OYN28CjzZkH57bh4w0I2F7IodpQvUAEd1mbWXg==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"openharmony"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/sunos-x64": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.27.4.tgz",
"integrity": "sha512-/gOzgaewZJfeJTlsWhvUEmUG4tWEY2Spp5M20INYRg2ZKl9QPO3QEEgPeRtLjEWSW8FilRNacPOg8R1uaYkA6g==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"sunos"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-arm64": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.27.4.tgz",
"integrity": "sha512-Z9SExBg2y32smoDQdf1HRwHRt6vAHLXcxD2uGgO/v2jK7Y718Ix4ndsbNMU/+1Qiem9OiOdaqitioZwxivhXYg==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-ia32": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.27.4.tgz",
"integrity": "sha512-DAyGLS0Jz5G5iixEbMHi5KdiApqHBWMGzTtMiJ72ZOLhbu/bzxgAe8Ue8CTS3n3HbIUHQz/L51yMdGMeoxXNJw==",
"cpu": [
"ia32"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-x64": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.27.4.tgz",
"integrity": "sha512-+knoa0BDoeXgkNvvV1vvbZX4+hizelrkwmGJBdT17t8FNPwG2lKemmuMZlmaNQ3ws3DKKCxpb4zRZEIp3UxFCg==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@swc/helpers": {
"version": "0.5.20",
"resolved": "https://registry.npmjs.org/@swc/helpers/-/helpers-0.5.20.tgz",
"integrity": "sha512-2egEBHUMasdypIzrprsu8g+OEVd7Vp2MM3a2eVlM/cyFYto0nGz5BX5BTgh/ShZZI9ed+ozEq+Ngt+rgmUs8tw==",
"license": "Apache-2.0",
"dependencies": {
"tslib": "^2.8.0"
}
},
"node_modules/@types/minimist": {
"version": "1.2.5",
"resolved": "https://registry.npmjs.org/@types/minimist/-/minimist-1.2.5.tgz",
"integrity": "sha512-hov8bUuiLiyFPGyFPE1lwWhmzYbirOXQNNo40+y3zow8aFVTeyn3VWL0VFFfdNddA8S4Vf0Tc062rzyNr7Paag==",
"dev": true,
"license": "MIT"
},
"node_modules/@types/node": {
"version": "25.5.0",
"resolved": "https://registry.npmjs.org/@types/node/-/node-25.5.0.tgz",
"integrity": "sha512-jp2P3tQMSxWugkCUKLRPVUpGaL5MVFwF8RDuSRztfwgN1wmqJeMSbKlnEtQqU8UrhTmzEmZdu2I6v2dpp7XIxw==",
"dev": true,
"license": "MIT",
"dependencies": {
"undici-types": "~7.18.0"
}
},
"node_modules/@types/pdfkit": {
"version": "0.17.5",
"resolved": "https://registry.npmjs.org/@types/pdfkit/-/pdfkit-0.17.5.tgz",
"integrity": "sha512-T3ZHnvF91HsEco5ClhBCOuBwobZfPcI2jaiSHybkkKYq4KhVIIurod94JVKvDIG0JXT6o3KiERC0X0//m8dyrg==",
"dev": true,
"license": "MIT",
"dependencies": {
"@types/node": "*"
}
},
"node_modules/base64-js": {
"version": "1.5.1",
"resolved": "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz",
"integrity": "sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/feross"
},
{
"type": "patreon",
"url": "https://www.patreon.com/feross"
},
{
"type": "consulting",
"url": "https://feross.org/support"
}
],
"license": "MIT"
},
"node_modules/brotli": {
"version": "1.3.3",
"resolved": "https://registry.npmjs.org/brotli/-/brotli-1.3.3.tgz",
"integrity": "sha512-oTKjJdShmDuGW94SyyaoQvAjf30dZaHnjJ8uAF+u2/vGJkJbJPJAT1gDiOJP5v1Zb6f9KEyW/1HpuaWIXtGHPg==",
"license": "MIT",
"dependencies": {
"base64-js": "^1.1.2"
}
},
"node_modules/clone": {
"version": "2.1.2",
"resolved": "https://registry.npmjs.org/clone/-/clone-2.1.2.tgz",
"integrity": "sha512-3Pe/CF1Nn94hyhIYpjtiLhdCoEoz0DqQ+988E9gmeEdQZlojxnOb74wctFyuwWQHzqyf9X7C7MG8juUpqBJT8w==",
"license": "MIT",
"engines": {
"node": ">=0.8"
}
},
"node_modules/crypto-js": {
"version": "4.2.0",
"resolved": "https://registry.npmjs.org/crypto-js/-/crypto-js-4.2.0.tgz",
"integrity": "sha512-KALDyEYgpY+Rlob/iriUtjV6d5Eq+Y191A5g4UqLAi8CyGP9N1+FdVbkc1SxKc2r4YAYqG8JzO2KGL+AizD70Q==",
"license": "MIT"
},
"node_modules/dfa": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/dfa/-/dfa-1.2.0.tgz",
"integrity": "sha512-ED3jP8saaweFTjeGX8HQPjeC1YYyZs98jGNZx6IiBvxW7JG5v492kamAQB3m2wop07CvU/RQmzcKr6bgcC5D/Q==",
"license": "MIT"
},
"node_modules/esbuild": {
"version": "0.27.4",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.27.4.tgz",
"integrity": "sha512-Rq4vbHnYkK5fws5NF7MYTU68FPRE1ajX7heQ/8QXXWqNgqqJ/GkmmyxIzUnf2Sr/bakf8l54716CcMGHYhMrrQ==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
"bin": {
"esbuild": "bin/esbuild"
},
"engines": {
"node": ">=18"
},
"optionalDependencies": {
"@esbuild/aix-ppc64": "0.27.4",
"@esbuild/android-arm": "0.27.4",
"@esbuild/android-arm64": "0.27.4",
"@esbuild/android-x64": "0.27.4",
"@esbuild/darwin-arm64": "0.27.4",
"@esbuild/darwin-x64": "0.27.4",
"@esbuild/freebsd-arm64": "0.27.4",
"@esbuild/freebsd-x64": "0.27.4",
"@esbuild/linux-arm": "0.27.4",
"@esbuild/linux-arm64": "0.27.4",
"@esbuild/linux-ia32": "0.27.4",
"@esbuild/linux-loong64": "0.27.4",
"@esbuild/linux-mips64el": "0.27.4",
"@esbuild/linux-ppc64": "0.27.4",
"@esbuild/linux-riscv64": "0.27.4",
"@esbuild/linux-s390x": "0.27.4",
"@esbuild/linux-x64": "0.27.4",
"@esbuild/netbsd-arm64": "0.27.4",
"@esbuild/netbsd-x64": "0.27.4",
"@esbuild/openbsd-arm64": "0.27.4",
"@esbuild/openbsd-x64": "0.27.4",
"@esbuild/openharmony-arm64": "0.27.4",
"@esbuild/sunos-x64": "0.27.4",
"@esbuild/win32-arm64": "0.27.4",
"@esbuild/win32-ia32": "0.27.4",
"@esbuild/win32-x64": "0.27.4"
}
},
"node_modules/fast-deep-equal": {
"version": "3.1.3",
"resolved": "https://registry.npmjs.org/fast-deep-equal/-/fast-deep-equal-3.1.3.tgz",
"integrity": "sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q==",
"license": "MIT"
},
"node_modules/fontkit": {
"version": "2.0.4",
"resolved": "https://registry.npmjs.org/fontkit/-/fontkit-2.0.4.tgz",
"integrity": "sha512-syetQadaUEDNdxdugga9CpEYVaQIxOwk7GlwZWWZ19//qW4zE5bknOKeMBDYAASwnpaSHKJITRLMF9m1fp3s6g==",
"license": "MIT",
"dependencies": {
"@swc/helpers": "^0.5.12",
"brotli": "^1.3.2",
"clone": "^2.1.2",
"dfa": "^1.2.0",
"fast-deep-equal": "^3.1.3",
"restructure": "^3.0.0",
"tiny-inflate": "^1.0.3",
"unicode-properties": "^1.4.0",
"unicode-trie": "^2.0.0"
}
},
"node_modules/fsevents": {
"version": "2.3.3",
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz",
"integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
}
},
"node_modules/get-tsconfig": {
"version": "4.13.7",
"resolved": "https://registry.npmjs.org/get-tsconfig/-/get-tsconfig-4.13.7.tgz",
"integrity": "sha512-7tN6rFgBlMgpBML5j8typ92BKFi2sFQvIdpAqLA2beia5avZDrMs0FLZiM5etShWq5irVyGcGMEA1jcDaK7A/Q==",
"dev": true,
"license": "MIT",
"dependencies": {
"resolve-pkg-maps": "^1.0.0"
},
"funding": {
"url": "https://github.com/privatenumber/get-tsconfig?sponsor=1"
}
},
"node_modules/jpeg-exif": {
"version": "1.1.4",
"resolved": "https://registry.npmjs.org/jpeg-exif/-/jpeg-exif-1.1.4.tgz",
"integrity": "sha512-a+bKEcCjtuW5WTdgeXFzswSrdqi0jk4XlEtZlx5A94wCoBpFjfFTbo/Tra5SpNCl/YFZPvcV1dJc+TAYeg6ROQ==",
"deprecated": "Package no longer supported. Contact Support at https://www.npmjs.com/support for more info.",
"license": "MIT"
},
"node_modules/linebreak": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/linebreak/-/linebreak-1.1.0.tgz",
"integrity": "sha512-MHp03UImeVhB7XZtjd0E4n6+3xr5Dq/9xI/5FptGk5FrbDR3zagPa2DS6U8ks/3HjbKWG9Q1M2ufOzxV2qLYSQ==",
"license": "MIT",
"dependencies": {
"base64-js": "0.0.8",
"unicode-trie": "^2.0.0"
}
},
"node_modules/linebreak/node_modules/base64-js": {
"version": "0.0.8",
"resolved": "https://registry.npmjs.org/base64-js/-/base64-js-0.0.8.tgz",
"integrity": "sha512-3XSA2cR/h/73EzlXXdU6YNycmYI7+kicTxks4eJg2g39biHR84slg2+des+p7iHYhbRg/udIS4TD53WabcOUkw==",
"license": "MIT",
"engines": {
"node": ">= 0.4"
}
},
"node_modules/minimist": {
"version": "1.2.8",
"resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz",
"integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==",
"license": "MIT",
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/pako": {
"version": "0.2.9",
"resolved": "https://registry.npmjs.org/pako/-/pako-0.2.9.tgz",
"integrity": "sha512-NUcwaKxUxWrZLpDG+z/xZaCgQITkA/Dv4V/T6bw7VON6l1Xz/VnrBqrYjZQ12TamKHzITTfOEIYUj48y2KXImA==",
"license": "MIT"
},
"node_modules/pdfkit": {
"version": "0.17.2",
"resolved": "https://registry.npmjs.org/pdfkit/-/pdfkit-0.17.2.tgz",
"integrity": "sha512-UnwF5fXy08f0dnp4jchFYAROKMNTaPqb/xgR8GtCzIcqoTnbOqtp3bwKvO4688oHI6vzEEs8Q6vqqEnC5IUELw==",
"license": "MIT",
"dependencies": {
"crypto-js": "^4.2.0",
"fontkit": "^2.0.4",
"jpeg-exif": "^1.1.4",
"linebreak": "^1.1.0",
"png-js": "^1.0.0"
}
},
"node_modules/png-js": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/png-js/-/png-js-1.0.0.tgz",
"integrity": "sha512-k+YsbhpA9e+EFfKjTCH3VW6aoKlyNYI6NYdTfDL4CIvFnvsuO84ttonmZE7rc+v23SLTH8XX+5w/Ak9v0xGY4g=="
},
"node_modules/resolve-pkg-maps": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/resolve-pkg-maps/-/resolve-pkg-maps-1.0.0.tgz",
"integrity": "sha512-seS2Tj26TBVOC2NIc2rOe2y2ZO7efxITtLZcGSOnHHNOQ7CkiUBfw0Iw2ck6xkIhPwLhKNLS8BO+hEpngQlqzw==",
"dev": true,
"license": "MIT",
"funding": {
"url": "https://github.com/privatenumber/resolve-pkg-maps?sponsor=1"
}
},
"node_modules/restructure": {
"version": "3.0.2",
"resolved": "https://registry.npmjs.org/restructure/-/restructure-3.0.2.tgz",
"integrity": "sha512-gSfoiOEA0VPE6Tukkrr7I0RBdE0s7H1eFCDBk05l1KIQT1UIKNc5JZy6jdyW6eYH3aR3g5b3PuL77rq0hvwtAw==",
"license": "MIT"
},
"node_modules/tiny-inflate": {
"version": "1.0.3",
"resolved": "https://registry.npmjs.org/tiny-inflate/-/tiny-inflate-1.0.3.tgz",
"integrity": "sha512-pkY1fj1cKHb2seWDy0B16HeWyczlJA9/WW3u3c4z/NiWDsO3DOU5D7nhTLE9CF0yXv/QZFY7sEJmj24dK+Rrqw==",
"license": "MIT"
},
"node_modules/tslib": {
"version": "2.8.1",
"resolved": "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz",
"integrity": "sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==",
"license": "0BSD"
},
"node_modules/tsx": {
"version": "4.21.0",
"resolved": "https://registry.npmjs.org/tsx/-/tsx-4.21.0.tgz",
"integrity": "sha512-5C1sg4USs1lfG0GFb2RLXsdpXqBSEhAaA/0kPL01wxzpMqLILNxIxIOKiILz+cdg/pLnOUxFYOR5yhHU666wbw==",
"dev": true,
"license": "MIT",
"dependencies": {
"esbuild": "~0.27.0",
"get-tsconfig": "^4.7.5"
},
"bin": {
"tsx": "dist/cli.mjs"
},
"engines": {
"node": ">=18.0.0"
},
"optionalDependencies": {
"fsevents": "~2.3.3"
}
},
"node_modules/typescript": {
"version": "5.9.3",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz",
"integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"tsc": "bin/tsc",
"tsserver": "bin/tsserver"
},
"engines": {
"node": ">=14.17"
}
},
"node_modules/undici-types": {
"version": "7.18.2",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-7.18.2.tgz",
"integrity": "sha512-AsuCzffGHJybSaRrmr5eHr81mwJU3kjw6M+uprWvCXiNeN9SOGwQ3Jn8jb8m3Z6izVgknn1R0FTCEAP2QrLY/w==",
"dev": true,
"license": "MIT"
},
"node_modules/unicode-properties": {
"version": "1.4.1",
"resolved": "https://registry.npmjs.org/unicode-properties/-/unicode-properties-1.4.1.tgz",
"integrity": "sha512-CLjCCLQ6UuMxWnbIylkisbRj31qxHPAurvena/0iwSVbQ2G1VY5/HjV0IRabOEbDHlzZlRdCrD4NhB0JtU40Pg==",
"license": "MIT",
"dependencies": {
"base64-js": "^1.3.0",
"unicode-trie": "^2.0.0"
}
},
"node_modules/unicode-trie": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/unicode-trie/-/unicode-trie-2.0.0.tgz",
"integrity": "sha512-x7bc76x0bm4prf1VLg79uhAzKw8DVboClSN5VxJuQ+LKDOVEW9CdH+VY7SP+vX7xCYQqzzgQpFqz15zeLvAtZQ==",
"license": "MIT",
"dependencies": {
"pako": "^0.2.5",
"tiny-inflate": "^1.0.0"
}
}
}
}
+23
View File
@@ -0,0 +1,23 @@
{
"name": "flight-finder-scripts",
"version": "1.0.0",
"description": "Flight finder helpers for request normalization, readiness gating, and fixed-template PDF rendering",
"type": "module",
"scripts": {
"normalize-request": "tsx src/cli.ts normalize-request",
"report-status": "tsx src/cli.ts report-status",
"render-report": "tsx src/cli.ts render-report",
"delivery-plan": "tsx src/cli.ts delivery-plan",
"test": "node --import tsx --test tests/*.test.ts"
},
"dependencies": {
"minimist": "^1.2.8",
"pdfkit": "^0.17.2"
},
"devDependencies": {
"@types/minimist": "^1.2.5",
"@types/pdfkit": "^0.17.3",
"tsx": "^4.20.6",
"typescript": "^5.9.2"
}
}
@@ -0,0 +1,75 @@
# Flight Finder Source Viability
This note records bounded live checks for the first implementation pass.
It is intentionally operational, not aspirational: if a source is blocked here, the skill must treat it as blocked unless later evidence replaces this note.
## Required sources
- KAYAK
- Skyscanner
- Expedia
- airline direct-booking cross-check
## Status
Checked on `2026-03-30` from Stefano's MacBook Air, with no VPN active, using the existing local `web-automation` / CloakBrowser probe scripts.
Route used for bounded checks:
- `DFW -> BLQ`
- outbound date: `2026-05-30`
- travelers: `3 adults`
### KAYAK
- Status: `viable`
- Probe: `node flight_kayak_sweep.mjs DFW BLQ 3 /tmp/flight-finder-kayak-dates.json ...`
- Evidence:
- title returned as `DFW to BLQ, 5/30`
- results included multiple one-stop itineraries with parsed USD fares such as `$877`, `$949`, `$955`
- direct-booking hints were visible for British Airways on at least some results
- Implementation note:
- KAYAK can be a primary source in this first implementation pass
- parsed text is workable, but still brittle enough that bounded retries and status fallback remain necessary
### Skyscanner
- Status: `viable`
- Probe: `node tmp_skyscanner_probe.mjs 'https://www.skyscanner.com/transport/flights/dfw/blq/260530/?adultsv2=3&cabinclass=economy&rtn=0'`
- Evidence:
- title returned as `Cheap flights from Dallas to Bologna on Skyscanner`
- results page exposed concrete prices, total trip prices, stops, and itinerary text
- one-stop and multi-stop options were visible in the captured text
- Implementation note:
- Skyscanner is viable for bounded result capture in this first pass
- itinerary extraction should still be treated as text-scrape, not a stable API
### Expedia
- Status: `viable`
- Probe: `node tmp_expedia_probe.mjs 'https://www.expedia.com/Flights-Search?...'`
- Evidence:
- title returned as `DFW to BLQ flights`
- results page exposed current lowest price, airline/stops filters, and concrete per-traveler options such as `$877`, `$949`, `$961`
- Expedia text already surfaced some itinerary summaries in a report-friendly format
- Implementation note:
- Expedia is viable for bounded result capture in this first pass
- as with the other aggregators, source-specific timeouts and fallback rules are still required
### Airline direct-booking cross-check
- Status: `degraded`
- Probe: `node tmp_skyscanner_probe.mjs 'https://www.united.com/en/us/fsr/choose-flights?...'`
- Evidence:
- United's booking shell loaded and recognized the route / date context
- the search then returned `united.com was unable to complete your request. Please try again later.`
- Implementation note:
- direct-airline cross-checking remains in scope, but it should be treated as best-effort in the first pass
- when a direct site fails or refuses completion, the skill should record the failure explicitly instead of hanging or pretending a clean cross-check happened
## Scope decision for implementation pass 1
- Primary bounded search sources: `KAYAK`, `Skyscanner`, `Expedia`
- Direct-airline cross-check: `best-effort / degraded`
- The skill should continue if the direct-airline step fails, but the report must say that the direct cross-check was not fully completed
+86
View File
@@ -0,0 +1,86 @@
import fs from "node:fs/promises";
import minimist from "minimist";
import {
DFW_BLQ_2026_PROMPT_DRAFT,
normalizeFlightReportRequest
} from "./request-normalizer.js";
import { getFlightReportStatus } from "./report-status.js";
import { renderFlightReportPdf } from "./report-pdf.js";
import { buildLukeDeliveryPlan } from "./report-delivery.js";
import type { FlightReportPayload, FlightReportRequestDraft } from "./types.js";
function usage(): never {
throw new Error(`Usage:
flight-finder normalize-request --input "<request.json>"
flight-finder normalize-request --legacy-dfw-blq
flight-finder report-status --input "<report-payload.json>"
flight-finder render-report --input "<report-payload.json>" --output "<report.pdf>"
flight-finder delivery-plan --to "<recipient@example.com>" --subject "<subject>" --body "<body>" --attach "<report.pdf>"`);
}
async function readJsonFile<T>(filePath: string): Promise<T> {
return JSON.parse(await fs.readFile(filePath, "utf8")) as T;
}
async function main(): Promise<void> {
const argv = minimist(process.argv.slice(2), {
string: ["input", "output", "to", "subject", "body", "attach"],
boolean: ["legacy-dfw-blq"]
});
const command = argv._[0];
if (!command) {
usage();
}
if (command === "normalize-request") {
const draft = argv["legacy-dfw-blq"]
? DFW_BLQ_2026_PROMPT_DRAFT
: await readJsonFile<FlightReportRequestDraft>(argv.input || usage());
console.log(JSON.stringify(normalizeFlightReportRequest(draft), null, 2));
return;
}
if (command === "report-status") {
const payload = await readJsonFile<FlightReportPayload>(argv.input || usage());
console.log(
JSON.stringify(
getFlightReportStatus(normalizeFlightReportRequest(payload.request), payload),
null,
2
)
);
return;
}
if (command === "render-report") {
const payload = await readJsonFile<FlightReportPayload>(argv.input || usage());
const rendered = await renderFlightReportPdf(payload, argv.output || usage());
console.log(rendered);
return;
}
if (command === "delivery-plan") {
console.log(
JSON.stringify(
buildLukeDeliveryPlan({
recipientEmail: argv.to,
subject: argv.subject || "",
body: argv.body || "",
attachmentPath: argv.attach || usage()
}),
null,
2
)
);
return;
}
usage();
}
main().catch((error) => {
console.error(error instanceof Error ? error.message : String(error));
process.exitCode = 1;
});
@@ -0,0 +1,25 @@
export function normalizeMarketCountry(
value: string | null | undefined
): string | null | undefined {
const cleaned = typeof value === "string" ? value.trim() : undefined;
if (!cleaned) {
return undefined;
}
const normalized = cleaned.toUpperCase();
if (!/^[A-Z]{2}$/.test(normalized)) {
throw new Error(
`Invalid marketCountry "${value}". Use an ISO 3166-1 alpha-2 uppercase country code such as "TH" or "DE".`
);
}
return normalized;
}
export function isPlausibleEmail(value: string | null | undefined): boolean {
if (!value) {
return false;
}
return /^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(value);
}
@@ -0,0 +1,36 @@
export type NormalizePriceInput = {
amount: number;
currency: string;
exchangeRates?: Record<string, number>;
};
export type NormalizedPrice = {
currency: "USD";
amountUsd: number;
notes: string[];
};
export function normalizePriceToUsd(
input: NormalizePriceInput
): NormalizedPrice {
const currency = input.currency.trim().toUpperCase();
if (currency === "USD") {
return {
currency: "USD",
amountUsd: input.amount,
notes: ["Price already in USD."]
};
}
const rate = input.exchangeRates?.[currency];
if (typeof rate !== "number" || !Number.isFinite(rate) || rate <= 0) {
throw new Error(`Missing usable exchange rate for ${currency}.`);
}
const amountUsd = Math.round(input.amount * rate * 100) / 100;
return {
currency: "USD",
amountUsd,
notes: [`Converted ${currency} to USD using rate ${rate}.`]
};
}
@@ -0,0 +1,50 @@
import { isPlausibleEmail } from "./input-validation.js";
type DeliveryPlanInput = {
recipientEmail?: string | null;
subject: string;
body: string;
attachmentPath: string;
};
export type DeliveryPlan = {
ready: boolean;
needsRecipientEmail: boolean;
command: string | null;
sender: "luke@fiorinis.com";
recipientEmail: string | null;
};
function shellQuote(value: string): string {
return `'${value.replace(/'/g, `'\"'\"'`)}'`;
}
export function buildLukeDeliveryPlan(input: DeliveryPlanInput): DeliveryPlan {
const recipientEmail = input.recipientEmail?.trim() || null;
if (!recipientEmail || !isPlausibleEmail(recipientEmail)) {
return {
ready: false,
needsRecipientEmail: true,
command: null,
sender: "luke@fiorinis.com",
recipientEmail
};
}
const validRecipientEmail: string = recipientEmail;
const command = [
"zsh ~/.openclaw/workspace/bin/gog-luke gmail send",
`--to ${shellQuote(validRecipientEmail)}`,
`--subject ${shellQuote(input.subject)}`,
`--body ${shellQuote(input.body)}`,
`--attach ${shellQuote(input.attachmentPath)}`
].join(" ");
return {
ready: true,
needsRecipientEmail: false,
command,
sender: "luke@fiorinis.com",
recipientEmail: validRecipientEmail
};
}
+407
View File
@@ -0,0 +1,407 @@
import fs from "node:fs";
import path from "node:path";
import PDFDocument from "pdfkit";
import { getFlightReportStatus } from "./report-status.js";
import { normalizeFlightReportRequest } from "./request-normalizer.js";
import type {
FlightLegWindow,
FlightPassengerGroup,
FlightQuote,
FlightRecommendationOption,
FlightReportPayload
} from "./types.js";
export class ReportValidationError extends Error {}
function bulletLines(value: string[] | undefined, fallback = "Not provided."): string[] {
return value?.length ? value : [fallback];
}
function asLegLabelMap(legs: FlightLegWindow[]): Map<string, string> {
return new Map(legs.map((leg) => [leg.id, leg.label || `${leg.origin} -> ${leg.destination}`]));
}
function asPassengerGroupLabelMap(groups: FlightPassengerGroup[]): Map<string, string> {
return new Map(
groups.map((group) => [group.id, group.label || `${group.adults} adult${group.adults === 1 ? "" : "s"}`])
);
}
function formatUsd(value: number): string {
return new Intl.NumberFormat("en-US", {
style: "currency",
currency: "USD",
maximumFractionDigits: 0
}).format(value);
}
function formatDateWindow(leg: FlightLegWindow): string {
if (leg.earliest && leg.latest) {
return leg.earliest === leg.latest ? leg.earliest : `${leg.earliest} to ${leg.latest}`;
}
if (leg.relativeToLegId) {
const min = leg.minDaysAfter ?? "?";
const max = leg.maxDaysAfter ?? "?";
return `${min}-${max} days after ${leg.relativeToLegId}`;
}
return "Not provided";
}
function ensurePageSpace(doc: PDFKit.PDFDocument, requiredHeight: number): void {
const bottomLimit = doc.page.height - doc.page.margins.bottom;
if (doc.y + requiredHeight > bottomLimit) {
doc.addPage();
}
}
const WORKSPACE_REPORTS_PREFIX = "/Users/stefano/.openclaw/workspace/reports/";
export function formatArtifactsRoot(value: string): string {
const trimmed = String(value || "").trim();
if (!trimmed) return "Not provided";
if (trimmed.startsWith(WORKSPACE_REPORTS_PREFIX)) {
return `reports/${trimmed.slice(WORKSPACE_REPORTS_PREFIX.length)}`;
}
return path.basename(trimmed) || trimmed;
}
function drawSectionHeader(doc: PDFKit.PDFDocument, title: string): void {
ensurePageSpace(doc, 28);
const x = doc.page.margins.left;
const y = doc.y;
const width = doc.page.width - doc.page.margins.left - doc.page.margins.right;
const height = 18;
doc.save();
doc.roundedRect(x, y, width, height, 3).fill("#10384E");
doc
.fillColor("white")
.font("Helvetica-Bold")
.fontSize(12)
.text(title, x + 8, y + 4, { width: width - 16 });
doc.restore();
doc.moveDown(1.2);
}
function drawBulletList(doc: PDFKit.PDFDocument, items: string[] | undefined, fallback = "Not provided."): void {
const lines = bulletLines(items, fallback);
for (const line of lines) {
const lineHeight = doc.heightOfString(line, {
width: doc.page.width - doc.page.margins.left - doc.page.margins.right - 14,
lineGap: 2
});
ensurePageSpace(doc, lineHeight + 12);
const startY = doc.y;
doc.circle(doc.page.margins.left + 4, startY + 6, 1.5).fill("#10384E");
doc
.fillColor("#1E2329")
.font("Helvetica")
.fontSize(10.5)
.text(line, doc.page.margins.left + 14, startY, {
width: doc.page.width - doc.page.margins.left - doc.page.margins.right - 14,
lineGap: 2
});
doc.moveDown(0.35);
}
doc.moveDown(0.2);
}
type QuoteBookingLink = {
label: string;
text: string;
url: string | null;
};
export function describeQuoteBookingLinks(quote: FlightQuote): QuoteBookingLink[] {
return [
{
label: "Search / book this fare",
text: `Search / book this fare: ${quote.displayPriceUsd} via ${quote.source}`,
url: quote.bookingLink
},
{
label: "Direct airline booking",
text: quote.directBookingUrl
? `Direct airline booking: ${quote.airlineName || "Airline site"}`
: "Direct airline booking: not captured in this run",
url: quote.directBookingUrl || null
}
];
}
function drawQuoteBookingLinks(doc: PDFKit.PDFDocument, links: QuoteBookingLink[], left: number, width: number): void {
for (const linkItem of links) {
const lineHeight = Math.max(
doc.heightOfString(`${linkItem.label}:`, { width: 130 }),
doc.heightOfString(linkItem.text.replace(`${linkItem.label}: `, ""), {
width: width - 132,
lineGap: 1
})
);
ensurePageSpace(doc, lineHeight + 8);
const startY = doc.y;
doc
.fillColor("#1E2329")
.font("Helvetica-Bold")
.fontSize(9.5)
.text(`${linkItem.label}:`, left, startY, { width: 130 });
doc
.fillColor(linkItem.url ? "#0B5EA8" : "#4F5B66")
.font("Helvetica")
.fontSize(9.5)
.text(linkItem.text.replace(`${linkItem.label}: `, ""), left + 132, startY, {
width: width - 132,
lineGap: 1,
...(linkItem.url ? { link: linkItem.url, underline: true } : {})
});
doc.moveDown(0.25);
}
}
function drawKeyValueTable(doc: PDFKit.PDFDocument, rows: Array<[string, string]>): void {
const left = doc.page.margins.left;
const totalWidth = doc.page.width - doc.page.margins.left - doc.page.margins.right;
const keyWidth = 170;
const valueWidth = totalWidth - keyWidth;
for (const [key, value] of rows) {
const keyHeight = doc.heightOfString(key, { width: keyWidth - 12 });
const valueHeight = doc.heightOfString(value, { width: valueWidth - 12 });
const rowHeight = Math.max(keyHeight, valueHeight) + 12;
ensurePageSpace(doc, rowHeight + 8);
const startY = doc.y;
doc.save();
doc.rect(left, startY, keyWidth, rowHeight).fill("#EEF3F7");
doc.rect(left + keyWidth, startY, valueWidth, rowHeight).fill("#FFFFFF");
doc
.lineWidth(0.5)
.strokeColor("#C7D0D9")
.rect(left, startY, totalWidth, rowHeight)
.stroke();
doc.moveTo(left + keyWidth, startY).lineTo(left + keyWidth, startY + rowHeight).stroke();
doc.restore();
doc
.fillColor("#1E2329")
.font("Helvetica-Bold")
.fontSize(9.5)
.text(key, left + 6, startY + 6, { width: keyWidth - 12 });
doc
.fillColor("#1E2329")
.font("Helvetica")
.fontSize(9.5)
.text(value, left + keyWidth + 6, startY + 6, { width: valueWidth - 12 });
doc.y = startY + rowHeight;
}
doc.moveDown(0.8);
}
function ensureFreshSearch(payload: FlightReportPayload): void {
const searchExecution = payload.searchExecution;
if (!searchExecution?.freshSearch) {
throw new ReportValidationError(
"The flight report payload must come from a fresh search run. Cached or previously captured results are not allowed."
);
}
const startedAt = Date.parse(searchExecution.startedAt);
const completedAt = Date.parse(searchExecution.completedAt);
if (!Number.isFinite(startedAt) || !Number.isFinite(completedAt) || completedAt < startedAt) {
throw new ReportValidationError(
"The flight report payload must include a valid fresh-search time window."
);
}
if (!String(searchExecution.artifactsRoot || "").trim()) {
throw new ReportValidationError(
"The flight report payload must include the artifacts root for the fresh search run."
);
}
}
function drawItineraryCard(
doc: PDFKit.PDFDocument,
option: FlightRecommendationOption,
quotesById: Map<string, FlightQuote>,
legLabels: Map<string, string>,
passengerLabels: Map<string, string>
): void {
const left = doc.page.margins.left;
const width = doc.page.width - doc.page.margins.left - doc.page.margins.right;
const badge = option.backup ? "Backup option" : "Primary recommendation";
const quoteBlocks = option.quoteIds.flatMap((quoteId) => {
const quote = quotesById.get(quoteId);
if (!quote) {
return [
{
heading: `Missing quote reference: ${quoteId}`,
details: [],
links: []
}
];
}
const passengerLine = quote.passengerGroupIds
.map((groupId) => passengerLabels.get(groupId) || groupId)
.join(", ");
const legLine = legLabels.get(quote.legId) || quote.legId;
return [
{
heading: `${legLine}: ${quote.itinerarySummary} (${quote.displayPriceUsd})`,
details: [
`Passengers: ${passengerLine}`,
`Timing: ${quote.departureTimeLocal || "?"} -> ${quote.arrivalTimeLocal || "?"}`,
`Routing: ${quote.stopsText || "stops?"}; ${quote.layoverText || "layover?"}; ${quote.totalDurationText || "duration?"}`,
...(quote.notes || []).map((note) => `Note: ${note}`)
],
links: describeQuoteBookingLinks(quote)
}
];
});
ensurePageSpace(doc, 28);
const top = doc.y;
doc.save();
doc.roundedRect(left, top, width, 22, 4).fill(option.backup ? "#586E75" : "#1E6B52");
doc
.fillColor("white")
.font("Helvetica-Bold")
.fontSize(10.5)
.text(`${badge}: ${option.title}`, left + 10, top + 6.5, { width: width - 20 });
doc.restore();
doc.y = top + 28;
drawKeyValueTable(doc, [
["Recommendation total", formatUsd(option.totalPriceUsd)],
["Why it ranked here", option.rationale]
]);
for (const block of quoteBlocks) {
ensurePageSpace(doc, 22);
doc
.fillColor("#10384E")
.font("Helvetica-Bold")
.fontSize(10)
.text(block.heading, left, doc.y, {
width,
lineGap: 2
});
doc.moveDown(0.2);
drawBulletList(doc, block.details);
drawQuoteBookingLinks(doc, block.links, left, width);
doc.moveDown(0.5);
}
doc.moveDown(0.2);
}
export async function renderFlightReportPdf(
payload: FlightReportPayload,
outputPath: string
): Promise<string> {
const normalizedRequest = normalizeFlightReportRequest(payload.request);
const status = getFlightReportStatus(normalizedRequest, payload);
if (!status.pdfReady) {
throw new ReportValidationError(
"The flight report payload is still incomplete. Finish the report before generating the PDF."
);
}
ensureFreshSearch(payload);
await fs.promises.mkdir(path.dirname(outputPath), { recursive: true });
const doc = new PDFDocument({
size: "LETTER",
margin: 50,
info: {
Title: payload.request.tripName || "Flight report",
Author: "OpenClaw flight-finder"
}
});
const stream = fs.createWriteStream(outputPath);
doc.pipe(stream);
const request = normalizedRequest.request;
const legLabels = asLegLabelMap(request.legs);
const passengerLabels = asPassengerGroupLabelMap(request.passengerGroups);
const quotesById = new Map(payload.quotes.map((quote) => [quote.id, quote]));
try {
doc.fillColor("#10384E").font("Helvetica-Bold").fontSize(22).text(request.tripName || "Flight report");
doc.moveDown(0.2);
doc
.fillColor("#4F5B66")
.font("Helvetica")
.fontSize(10)
.text(`Generated: ${payload.generatedAt}`);
doc
.fillColor("#4F5B66")
.font("Helvetica")
.fontSize(10)
.text(
`Market: ${payload.marketCountryUsed || "Default market"} | Fresh search completed: ${payload.searchExecution.completedAt}`
);
doc.moveDown();
drawSectionHeader(doc, "Trip Snapshot");
drawKeyValueTable(doc, [
["Recipient", request.recipientEmail || "Missing"],
["Passenger groups", request.passengerGroups.map((group) => group.label || `${group.adults} adults`).join(" | ")],
["Preferences", [
request.preferences.flexibleDates ? "Flexible dates" : "Fixed dates",
request.preferences.maxStops != null ? `Max stops ${request.preferences.maxStops}` : "Stops not capped",
request.preferences.maxLayoverHours != null ? `Layover <= ${request.preferences.maxLayoverHours}h` : "Layover open"
].join(" | ")],
["Market-country mode", request.preferences.marketCountry || "Off"],
["Search artifacts", formatArtifactsRoot(payload.searchExecution.artifactsRoot)]
]);
drawSectionHeader(doc, "Legs And Travelers");
for (const leg of request.legs) {
const assignedGroups = request.legAssignments
.find((assignment) => assignment.legId === leg.id)
?.passengerGroupIds.map((groupId) => passengerLabels.get(groupId) || groupId)
.join(", ") || "Not assigned";
drawKeyValueTable(doc, [
["Leg", leg.label || leg.id],
["Route", `${leg.origin} -> ${leg.destination}`],
["Window", formatDateWindow(leg)],
["Travelers", assignedGroups]
]);
}
drawSectionHeader(doc, "Executive Summary");
drawBulletList(doc, payload.executiveSummary);
drawSectionHeader(doc, "Recommended Options");
payload.rankedOptions.forEach((option) => drawItineraryCard(doc, option, quotesById, legLabels, passengerLabels));
drawSectionHeader(doc, "Source Findings");
payload.sourceFindings.forEach((finding) => {
drawKeyValueTable(doc, [
["Source", finding.source],
["Status", finding.status],
["Checked", finding.checkedAt]
]);
drawBulletList(doc, finding.notes, "No source notes.");
});
drawSectionHeader(doc, "Warnings And Degraded Conditions");
drawBulletList(doc, [...payload.reportWarnings, ...payload.degradedReasons], "No additional warnings.");
doc.end();
await new Promise<void>((resolve, reject) => {
stream.on("finish", () => resolve());
stream.on("error", reject);
});
} catch (error) {
stream.destroy();
await fs.promises.unlink(outputPath).catch(() => {});
throw error;
}
return outputPath;
}
+83
View File
@@ -0,0 +1,83 @@
import type {
FlightReportPayload,
FlightReportStatusResult,
NormalizedFlightReportRequest
} from "./types.js";
import { isPlausibleEmail } from "./input-validation.js";
export function getFlightReportStatus(
normalizedRequest: NormalizedFlightReportRequest,
payload?: FlightReportPayload | null
): FlightReportStatusResult {
if (normalizedRequest.missingSearchInputs.length) {
return {
needsMissingInputs: normalizedRequest.missingSearchInputs,
readyToSearch: false,
pdfReady: false,
emailReady: false,
chatSummaryReady: false,
terminalOutcome: "missing-inputs",
degraded: false,
degradedReasons: [],
blockingReason: "Missing search-critical trip inputs."
};
}
if (!payload) {
return {
needsMissingInputs: normalizedRequest.missingDeliveryInputs,
readyToSearch: true,
pdfReady: false,
emailReady: false,
chatSummaryReady: false,
terminalOutcome: "report-incomplete",
degraded: false,
degradedReasons: [],
lastCompletedPhase: "intake",
blockingReason: "Search and report assembly have not completed yet."
};
}
const allSourcesFailed =
payload.sourceFindings.length > 0 &&
payload.sourceFindings.every((finding) => finding.status === "blocked") &&
payload.quotes.length === 0;
if (allSourcesFailed) {
return {
needsMissingInputs: normalizedRequest.missingDeliveryInputs,
readyToSearch: true,
pdfReady: false,
emailReady: false,
chatSummaryReady: true,
terminalOutcome: "all-sources-failed",
degraded: true,
degradedReasons: payload.degradedReasons,
lastCompletedPhase: payload.lastCompletedPhase || "search",
blockingReason: "All configured travel sources failed or were blocked."
};
}
const reportComplete = Boolean(
payload.quotes.length &&
payload.rankedOptions.length &&
payload.executiveSummary.length
);
const validRecipient = isPlausibleEmail(normalizedRequest.request.recipientEmail);
return {
needsMissingInputs: validRecipient ? [] : normalizedRequest.missingDeliveryInputs,
readyToSearch: true,
pdfReady: reportComplete,
emailReady: reportComplete && validRecipient,
chatSummaryReady:
payload.quotes.length > 0 ||
payload.rankedOptions.length > 0 ||
payload.executiveSummary.length > 0,
terminalOutcome: reportComplete ? "ready" : "report-incomplete",
degraded: payload.degradedReasons.length > 0,
degradedReasons: payload.degradedReasons,
lastCompletedPhase: payload.lastCompletedPhase,
blockingReason: reportComplete ? undefined : "The report payload is still incomplete."
};
}
@@ -0,0 +1,309 @@
import type {
FlightLegWindow,
FlightPassengerGroup,
FlightReportRequest,
FlightReportRequestDraft,
FlightSearchPreferences,
NormalizedFlightReportRequest
} from "./types.js";
import { isPlausibleEmail, normalizeMarketCountry } from "./input-validation.js";
const DEFAULT_PREFERENCES: FlightSearchPreferences = {
preferOneStop: true,
maxLayoverHours: 6,
normalizeCurrencyTo: "USD"
};
function cleanString(value: string | null | undefined): string | undefined {
if (typeof value !== "string") {
return undefined;
}
const trimmed = value.trim();
return trimmed ? trimmed : undefined;
}
function uniqueStrings(values: Array<string | null | undefined> | undefined): string[] | undefined {
if (!values?.length) {
return undefined;
}
const cleaned = values
.map((value) => cleanString(value))
.filter((value): value is string => Boolean(value));
return cleaned.length ? Array.from(new Set(cleaned)) : undefined;
}
function normalizePassengerGroups(
groups: FlightReportRequestDraft["passengerGroups"]
): FlightPassengerGroup[] {
return (groups || [])
.filter((group): group is NonNullable<typeof group> => Boolean(group))
.map((group, index) => ({
id: cleanString(group.id) || `group-${index + 1}`,
adults: Number(group.adults || 0),
children: Number(group.children || 0) || undefined,
infants: Number(group.infants || 0) || undefined,
label: cleanString(group.label)
}));
}
function normalizeLegs(legs: FlightReportRequestDraft["legs"]): FlightLegWindow[] {
return (legs || [])
.filter((leg): leg is NonNullable<typeof leg> => Boolean(leg))
.map((leg, index) => ({
id: cleanString(leg.id) || `leg-${index + 1}`,
origin: cleanString(leg.origin) || "",
destination: cleanString(leg.destination) || "",
earliest: cleanString(leg.earliest),
latest: cleanString(leg.latest),
relativeToLegId: cleanString(leg.relativeToLegId),
minDaysAfter:
typeof leg.minDaysAfter === "number" && Number.isFinite(leg.minDaysAfter)
? leg.minDaysAfter
: undefined,
maxDaysAfter:
typeof leg.maxDaysAfter === "number" && Number.isFinite(leg.maxDaysAfter)
? leg.maxDaysAfter
: undefined,
label: cleanString(leg.label)
}));
}
function normalizeAssignments(
draft: FlightReportRequestDraft["legAssignments"]
): FlightReportRequest["legAssignments"] {
return (draft || [])
.filter((assignment): assignment is NonNullable<typeof assignment> => Boolean(assignment))
.map((assignment) => ({
legId: cleanString(assignment.legId) || "",
passengerGroupIds:
uniqueStrings(
(assignment.passengerGroupIds || []).map((value) =>
typeof value === "string" ? value : undefined
)
) || []
}));
}
function normalizePreferences(
preferences: FlightReportRequestDraft["preferences"]
): FlightSearchPreferences {
const draft = preferences || {};
return {
...DEFAULT_PREFERENCES,
cabin: draft.cabin,
maxStops: typeof draft.maxStops === "number" ? draft.maxStops : undefined,
preferOneStop:
typeof draft.preferOneStop === "boolean"
? draft.preferOneStop
: DEFAULT_PREFERENCES.preferOneStop,
maxLayoverHours:
typeof draft.maxLayoverHours === "number"
? draft.maxLayoverHours
: DEFAULT_PREFERENCES.maxLayoverHours,
excludeAirlines: uniqueStrings(draft.excludeAirlines),
excludeCountries: uniqueStrings(draft.excludeCountries),
excludeAirports: uniqueStrings(draft.excludeAirports),
flexibleDates:
typeof draft.flexibleDates === "boolean" ? draft.flexibleDates : undefined,
requireAirlineDirectCrossCheck:
typeof draft.requireAirlineDirectCrossCheck === "boolean"
? draft.requireAirlineDirectCrossCheck
: undefined,
specialConstraints: uniqueStrings(draft.specialConstraints),
geoPricingMarket: cleanString(draft.geoPricingMarket),
marketCountry: normalizeMarketCountry(draft.marketCountry),
normalizeCurrencyTo: "USD"
};
}
function collectMissingSearchInputs(request: FlightReportRequest): string[] {
const missing = new Set<string>();
if (!request.legs.length) {
missing.add("trip legs");
}
if (!request.passengerGroups.length) {
missing.add("passenger groups");
}
for (const group of request.passengerGroups) {
const total = group.adults + (group.children || 0) + (group.infants || 0);
if (total <= 0) {
missing.add(`traveler count for passenger group ${group.label || group.id}`);
}
}
const legIds = new Set(request.legs.map((leg) => leg.id));
const groupIds = new Set(request.passengerGroups.map((group) => group.id));
for (const leg of request.legs) {
const label = leg.label || leg.id;
if (!leg.origin || !leg.destination) {
missing.add(`origin and destination for ${label}`);
}
const hasAbsoluteWindow = Boolean(leg.earliest || leg.latest);
const hasRelativeWindow = Boolean(
leg.relativeToLegId &&
(typeof leg.minDaysAfter === "number" || typeof leg.maxDaysAfter === "number")
);
if (!hasAbsoluteWindow && !hasRelativeWindow) {
missing.add(`date window for ${label}`);
}
if (leg.relativeToLegId && !legIds.has(leg.relativeToLegId)) {
missing.add(`valid reference leg for ${label}`);
}
}
if (!request.legAssignments.length) {
missing.add("passenger assignments for every leg");
}
for (const leg of request.legs) {
const label = leg.label || leg.id;
const assignment = request.legAssignments.find((entry) => entry.legId === leg.id);
if (!assignment) {
missing.add(`passenger assignments for ${label}`);
continue;
}
if (!assignment.passengerGroupIds.length) {
missing.add(`assigned travelers for ${label}`);
continue;
}
if (assignment.passengerGroupIds.some((groupId) => !groupIds.has(groupId))) {
missing.add(`valid passenger-group references for ${label}`);
}
}
return Array.from(missing);
}
export function normalizeFlightReportRequest(
draft: FlightReportRequestDraft
): NormalizedFlightReportRequest {
const normalizedRecipientEmail = cleanString(draft.recipientEmail) || null;
const request: FlightReportRequest = {
tripName: cleanString(draft.tripName),
legs: normalizeLegs(draft.legs),
passengerGroups: normalizePassengerGroups(draft.passengerGroups),
legAssignments: normalizeAssignments(draft.legAssignments),
recipientEmail: normalizedRecipientEmail,
preferences: normalizePreferences(draft.preferences)
};
const missingSearchInputs = collectMissingSearchInputs(request);
const missingDeliveryInputs = !request.recipientEmail
? ["recipient email"]
: isPlausibleEmail(request.recipientEmail)
? []
: ["valid recipient email"];
const warnings: string[] = [];
if (!request.recipientEmail) {
warnings.push(
"Recipient email is still missing. Ask for it before rendering or sending the PDF report."
);
} else if (!isPlausibleEmail(request.recipientEmail)) {
warnings.push(
"Recipient email looks malformed. Ask for a corrected email address before rendering or sending the PDF report."
);
}
if (request.preferences.marketCountry && !request.preferences.geoPricingMarket) {
warnings.push(
`Market-localized search is explicit for this run. Connect VPN to ${request.preferences.marketCountry} only for the bounded search phase, then disconnect before ranking/render/delivery.`
);
}
return {
request,
readyToSearch: missingSearchInputs.length === 0,
missingInputs: [...missingSearchInputs, ...missingDeliveryInputs],
missingSearchInputs,
missingDeliveryInputs,
warnings
};
}
export const DFW_BLQ_2026_PROMPT_DRAFT: FlightReportRequestDraft = {
tripName: "DFW ↔ BLQ flight report",
legs: [
{
id: "outbound",
origin: "DFW",
destination: "BLQ",
earliest: "2026-05-30",
latest: "2026-06-07",
label: "Outbound"
},
{
id: "return-pair",
origin: "BLQ",
destination: "DFW",
relativeToLegId: "outbound",
minDaysAfter: 6,
maxDaysAfter: 10,
label: "Return for 2 adults"
},
{
id: "return-solo",
origin: "BLQ",
destination: "DFW",
earliest: "2026-06-28",
latest: "2026-07-05",
label: "Return for 1 adult"
}
],
passengerGroups: [
{
id: "pair",
adults: 2,
label: "2 adults traveling together"
},
{
id: "solo",
adults: 1,
label: "1 adult returning separately"
}
],
legAssignments: [
{
legId: "outbound",
passengerGroupIds: ["pair", "solo"]
},
{
legId: "return-pair",
passengerGroupIds: ["pair"]
},
{
legId: "return-solo",
passengerGroupIds: ["solo"]
}
],
recipientEmail: "stefano@fiorinis.com",
preferences: {
preferOneStop: true,
maxStops: 1,
maxLayoverHours: 6,
flexibleDates: true,
excludeAirlines: ["Turkish Airlines"],
excludeCountries: ["TR"],
requireAirlineDirectCrossCheck: true,
specialConstraints: [
"Exclude itineraries operated by airlines based in Arab or Middle Eastern countries.",
"Exclude itineraries routing through airports in Arab or Middle Eastern countries.",
"Start from a fresh browser/session profile for this search."
],
geoPricingMarket: "Thailand",
marketCountry: "TH",
normalizeCurrencyTo: "USD"
}
};
+59
View File
@@ -0,0 +1,59 @@
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import type { FlightRunState } from "./types.js";
const DEFAULT_STATE_DIR = path.join(
os.homedir(),
".openclaw",
"workspace",
"state",
"flight-finder"
);
export function getFlightFinderStatePath(
baseDir = DEFAULT_STATE_DIR
): string {
return path.join(baseDir, "flight-finder-run.json");
}
export async function saveFlightFinderRunState(
state: FlightRunState,
baseDir = DEFAULT_STATE_DIR
): Promise<string> {
await fs.mkdir(baseDir, { recursive: true });
const statePath = getFlightFinderStatePath(baseDir);
await fs.writeFile(statePath, JSON.stringify(state, null, 2));
return statePath;
}
export async function loadFlightFinderRunState(
baseDir = DEFAULT_STATE_DIR
): Promise<FlightRunState | null> {
const statePath = getFlightFinderStatePath(baseDir);
try {
const raw = await fs.readFile(statePath, "utf8");
return JSON.parse(raw) as FlightRunState;
} catch (error) {
const maybeNodeError = error as NodeJS.ErrnoException;
if (maybeNodeError.code === "ENOENT") {
return null;
}
throw error;
}
}
export async function clearFlightFinderRunState(
baseDir = DEFAULT_STATE_DIR
): Promise<void> {
const statePath = getFlightFinderStatePath(baseDir);
try {
await fs.unlink(statePath);
} catch (error) {
const maybeNodeError = error as NodeJS.ErrnoException;
if (maybeNodeError.code !== "ENOENT") {
throw error;
}
}
}
@@ -0,0 +1,81 @@
import type {
FlightReportRequest,
FlightSearchPlan,
FlightSearchSourceFinding,
FlightSearchSourceName,
FlightSearchSourceViability
} from "./types.js";
export const SOURCE_SILENT_TIMEOUT_MS = 45_000;
export const SOURCE_TOTAL_TIMEOUT_MS = 180_000;
export const RUN_TOTAL_TIMEOUT_MS = 900_000;
export const VPN_CONNECT_TIMEOUT_MS = 30_000;
export const VPN_DISCONNECT_TIMEOUT_MS = 15_000;
const DEFAULT_SOURCE_ORDER: FlightSearchSourceName[] = [
"kayak",
"skyscanner",
"expedia",
"airline-direct"
];
function findingFor(
source: FlightSearchSourceName,
findings: FlightSearchSourceFinding[]
): FlightSearchSourceFinding | undefined {
return findings.find((entry) => entry.source === source);
}
function sourceStatus(
source: FlightSearchSourceName,
findings: FlightSearchSourceFinding[]
): FlightSearchSourceViability {
return findingFor(source, findings)?.status || "viable";
}
export function buildFlightSearchPlan(
request: FlightReportRequest,
findings: FlightSearchSourceFinding[] = []
): FlightSearchPlan {
const sourceOrder = DEFAULT_SOURCE_ORDER.map((source) => {
const status = sourceStatus(source, findings);
const finding = findingFor(source, findings);
const required = source !== "airline-direct";
const enabled = status !== "blocked";
const reason =
finding?.notes[0] ||
(source === "airline-direct"
? "Best-effort airline direct cross-check."
: "Primary bounded search source.");
return {
source,
enabled,
required,
status,
silentTimeoutMs: SOURCE_SILENT_TIMEOUT_MS,
totalTimeoutMs: SOURCE_TOTAL_TIMEOUT_MS,
reason
};
});
const degradedReasons = sourceOrder
.filter((source) => source.status !== "viable")
.map((source) =>
source.enabled
? `${source.source} is degraded for this run: ${source.reason}`
: `${source.source} is blocked for this run: ${source.reason}`
);
return {
sourceOrder,
vpn: {
enabled: Boolean(request.preferences.marketCountry),
marketCountry: request.preferences.marketCountry || null,
connectTimeoutMs: VPN_CONNECT_TIMEOUT_MS,
disconnectTimeoutMs: VPN_DISCONNECT_TIMEOUT_MS,
fallbackMode: "default-market"
},
degradedReasons
};
}
+194
View File
@@ -0,0 +1,194 @@
export type FlightPassengerGroup = {
id: string;
adults: number;
children?: number;
infants?: number;
label?: string;
};
export type FlightLegWindow = {
id: string;
origin: string;
destination: string;
earliest?: string;
latest?: string;
relativeToLegId?: string;
minDaysAfter?: number;
maxDaysAfter?: number;
label?: string;
};
export type FlightSearchPreferences = {
cabin?: "economy" | "premium-economy" | "business" | "first";
maxStops?: number;
preferOneStop?: boolean;
maxLayoverHours?: number;
excludeAirlines?: string[];
excludeCountries?: string[];
excludeAirports?: string[];
flexibleDates?: boolean;
requireAirlineDirectCrossCheck?: boolean;
specialConstraints?: string[];
geoPricingMarket?: string | null;
marketCountry?: string | null;
normalizeCurrencyTo?: "USD";
};
export type FlightReportRequest = {
tripName?: string;
legs: FlightLegWindow[];
passengerGroups: FlightPassengerGroup[];
legAssignments: Array<{ legId: string; passengerGroupIds: string[] }>;
recipientEmail?: string | null;
preferences: FlightSearchPreferences;
};
export type FlightReportStatus = {
needsMissingInputs: string[];
readyToSearch: boolean;
pdfReady: boolean;
emailReady: boolean;
lastCompletedPhase?: "intake" | "search" | "ranking" | "render" | "delivery";
};
export type FlightQuote = {
id: string;
source: FlightSearchSourceName;
legId: string;
passengerGroupIds: string[];
bookingLink: string;
itinerarySummary: string;
airlineName?: string;
departureTimeLocal?: string;
arrivalTimeLocal?: string;
stopsText?: string;
layoverText?: string;
totalDurationText?: string;
totalPriceUsd: number;
displayPriceUsd: string;
originalCurrency?: string;
originalTotalPrice?: number;
directBookingUrl?: string | null;
crossCheckStatus?: "verified" | "not-available" | "failed";
notes?: string[];
};
export type FlightRecommendationOption = {
id: string;
title: string;
quoteIds: string[];
totalPriceUsd: number;
rationale: string;
backup?: boolean;
};
export type FlightSearchSourceName =
| "kayak"
| "skyscanner"
| "expedia"
| "airline-direct";
export type FlightSearchSourceViability = "viable" | "degraded" | "blocked";
export type FlightSearchSourceFinding = {
source: FlightSearchSourceName;
status: FlightSearchSourceViability;
checkedAt: string;
notes: string[];
evidence?: string[];
};
export type FlightSearchPlanSource = {
source: FlightSearchSourceName;
enabled: boolean;
required: boolean;
status: FlightSearchSourceViability;
silentTimeoutMs: number;
totalTimeoutMs: number;
reason: string;
};
export type FlightSearchPlan = {
sourceOrder: FlightSearchPlanSource[];
vpn: {
enabled: boolean;
marketCountry: string | null;
connectTimeoutMs: number;
disconnectTimeoutMs: number;
fallbackMode: "default-market";
};
degradedReasons: string[];
};
export type FlightReportRequestDraft = {
tripName?: string | null;
legs?: Array<Partial<FlightLegWindow> | null | undefined> | null;
passengerGroups?: Array<Partial<FlightPassengerGroup> | null | undefined> | null;
legAssignments?:
| Array<
| {
legId?: string | null;
passengerGroupIds?: Array<string | null | undefined> | null;
}
| null
| undefined
>
| null;
recipientEmail?: string | null;
preferences?: Partial<FlightSearchPreferences> | null;
};
export type NormalizedFlightReportRequest = {
request: FlightReportRequest;
readyToSearch: boolean;
missingInputs: string[];
missingSearchInputs: string[];
missingDeliveryInputs: string[];
warnings: string[];
};
export type FlightReportPayload = {
request: FlightReportRequest;
searchExecution: {
freshSearch: boolean;
startedAt: string;
completedAt: string;
artifactsRoot: string;
notes?: string[];
};
sourceFindings: FlightSearchSourceFinding[];
quotes: FlightQuote[];
rankedOptions: FlightRecommendationOption[];
executiveSummary: string[];
reportWarnings: string[];
degradedReasons: string[];
comparisonCurrency: "USD";
marketCountryUsed?: string | null;
lastCompletedPhase?: "intake" | "search" | "ranking" | "render" | "delivery";
renderedPdfPath?: string | null;
deliveredTo?: string | null;
emailAuthorized?: boolean;
generatedAt: string;
};
export type FlightReportStatusResult = FlightReportStatus & {
chatSummaryReady: boolean;
terminalOutcome:
| "ready"
| "missing-inputs"
| "all-sources-failed"
| "report-incomplete";
degraded: boolean;
degradedReasons: string[];
blockingReason?: string;
};
export type FlightRunState = {
request: FlightReportRequest;
lastCompletedPhase: "intake" | "search" | "ranking" | "render" | "delivery";
updatedAt: string;
reportWarnings: string[];
degradedReasons: string[];
sourceFindings: FlightSearchSourceFinding[];
comparisonCurrency: "USD";
};
@@ -0,0 +1,38 @@
import test from "node:test";
import assert from "node:assert/strict";
import { normalizePriceToUsd } from "../src/price-normalization.js";
test("normalizePriceToUsd passes through USD values", () => {
const result = normalizePriceToUsd({
amount: 2847,
currency: "USD"
});
assert.equal(result.amountUsd, 2847);
assert.equal(result.currency, "USD");
});
test("normalizePriceToUsd converts foreign currencies when a rate is supplied", () => {
const result = normalizePriceToUsd({
amount: 100,
currency: "EUR",
exchangeRates: {
EUR: 1.08
}
});
assert.equal(result.amountUsd, 108);
assert.match(result.notes.join(" "), /EUR/i);
});
test("normalizePriceToUsd rejects missing exchange rates", () => {
assert.throws(
() =>
normalizePriceToUsd({
amount: 100,
currency: "EUR"
}),
/exchange rate/i
);
});
@@ -0,0 +1,44 @@
import test from "node:test";
import assert from "node:assert/strict";
import { buildLukeDeliveryPlan } from "../src/report-delivery.js";
test("buildLukeDeliveryPlan requires a plausible recipient email", () => {
const plan = buildLukeDeliveryPlan({
recipientEmail: "invalid-email",
subject: "DFW ↔ BLQ flight report",
body: "Summary",
attachmentPath: "/tmp/report.pdf"
});
assert.equal(plan.ready, false);
assert.equal(plan.needsRecipientEmail, true);
assert.equal(plan.command, null);
});
test("buildLukeDeliveryPlan uses Luke's wrapper path when recipient is valid", () => {
const plan = buildLukeDeliveryPlan({
recipientEmail: "stefano@fiorinis.com",
subject: "DFW ↔ BLQ flight report",
body: "Summary",
attachmentPath: "/tmp/report.pdf"
});
assert.equal(plan.ready, true);
assert.match(String(plan.command), /gog-luke gmail send/);
assert.match(String(plan.command), /stefano@fiorinis.com/);
assert.match(String(plan.command), /report\.pdf/);
});
test("buildLukeDeliveryPlan safely quotes shell-sensitive content", () => {
const plan = buildLukeDeliveryPlan({
recipientEmail: "stefano@fiorinis.com",
subject: "Luke's flight report; rm -rf /",
body: "It's ready.",
attachmentPath: "/tmp/report's-final.pdf"
});
assert.equal(plan.ready, true);
assert.match(String(plan.command), /--subject 'Luke'"'"'s flight report; rm -rf \/'/);
assert.match(String(plan.command), /--attach '\/tmp\/report'"'"'s-final\.pdf'/);
});
@@ -0,0 +1,273 @@
import test from "node:test";
import assert from "node:assert/strict";
import os from "node:os";
import path from "node:path";
import fs from "node:fs";
import {
describeQuoteBookingLinks,
formatArtifactsRoot,
renderFlightReportPdf,
ReportValidationError
} from "../src/report-pdf.js";
import type { FlightReportPayload } from "../src/types.js";
import {
DFW_BLQ_2026_PROMPT_DRAFT,
normalizeFlightReportRequest
} from "../src/request-normalizer.js";
function samplePayload(): FlightReportPayload {
return {
request: normalizeFlightReportRequest(DFW_BLQ_2026_PROMPT_DRAFT).request,
searchExecution: {
freshSearch: true,
startedAt: "2026-03-30T20:45:00Z",
completedAt: "2026-03-30T21:00:00Z",
artifactsRoot: "/Users/stefano/.openclaw/workspace/reports/dfw-blq-flight-report-2026-03-30-exec",
notes: ["Fresh bounded search executed for this report."]
},
sourceFindings: [
{
source: "kayak",
status: "viable",
checkedAt: "2026-03-30T21:00:00Z",
notes: ["Returned usable options."]
}
],
quotes: [
{
id: "quote-1",
source: "kayak",
legId: "outbound",
passengerGroupIds: ["pair", "solo"],
bookingLink: "https://example.com/quote-1",
itinerarySummary: "DFW -> BLQ via LHR",
totalPriceUsd: 2847,
displayPriceUsd: "$2,847"
}
],
rankedOptions: [
{
id: "primary",
title: "Best overall itinerary",
quoteIds: ["quote-1"],
totalPriceUsd: 2847,
rationale: "Best price-to-convenience tradeoff."
}
],
executiveSummary: ["Best fare currently found is $2,847 total for 3 adults."],
reportWarnings: [],
degradedReasons: [],
comparisonCurrency: "USD",
generatedAt: "2026-03-30T21:00:00Z",
lastCompletedPhase: "ranking"
};
}
function countPdfPages(outputPath: string): number {
return fs.readFileSync(outputPath, "latin1").split("/Type /Page").length - 1;
}
test("renderFlightReportPdf writes a non-empty PDF", async () => {
const outputPath = path.join(os.tmpdir(), `flight-finder-${Date.now()}.pdf`);
await renderFlightReportPdf(samplePayload(), outputPath);
assert.ok(fs.existsSync(outputPath));
assert.ok(fs.statSync(outputPath).size > 0);
});
test("renderFlightReportPdf embeds booking links and only includes direct-airline URLs when captured", async () => {
const outputPath = path.join(os.tmpdir(), `flight-finder-links-${Date.now()}.pdf`);
const payload = samplePayload();
payload.quotes = [
{
...payload.quotes[0],
bookingLink: "https://example.com/quote-1",
directBookingUrl: null
},
{
id: "quote-2",
source: "airline-direct",
legId: "return-pair",
passengerGroupIds: ["pair"],
bookingLink: "https://example.com/quote-2",
directBookingUrl: "https://www.britishairways.com/",
airlineName: "British Airways",
itinerarySummary: "BLQ -> DFW via LHR",
totalPriceUsd: 1100,
displayPriceUsd: "$1,100"
}
];
payload.rankedOptions = [
{
id: "primary",
title: "Best overall itinerary",
quoteIds: ["quote-1", "quote-2"],
totalPriceUsd: 3947,
rationale: "Best price-to-convenience tradeoff."
}
];
await renderFlightReportPdf(payload, outputPath);
const pdfContents = fs.readFileSync(outputPath, "latin1");
assert.match(pdfContents, /https:\/\/example\.com\/quote-1/);
assert.match(pdfContents, /https:\/\/example\.com\/quote-2/);
assert.match(pdfContents, /https:\/\/www\.britishairways\.com\//);
assert.doesNotMatch(pdfContents, /Direct airline booking: not captured in this run[\s\S]*https:\/\/example\.com\/quote-1/);
});
test("renderFlightReportPdf avoids raw absolute artifact paths and keeps page count bounded for a long report", async () => {
const outputPath = path.join(os.tmpdir(), `flight-finder-long-${Date.now()}.pdf`);
const payload = samplePayload();
payload.searchExecution.artifactsRoot =
"/Users/stefano/.openclaw/workspace/reports/dfw-blq-flight-report-2026-03-30-fresh-185742";
payload.quotes = [
{
...payload.quotes[0],
id: "quote-1",
notes: [
"Observed on fresh KAYAK sweep at $794 per traveler for 3 adults.",
"Skyscanner TH/USD corroborated the same Madrid pattern.",
"Expedia also found a nearby fare band."
]
},
{
id: "quote-2",
source: "kayak",
legId: "return-pair",
passengerGroupIds: ["pair"],
bookingLink: "https://example.com/quote-2",
itinerarySummary: "BLQ -> DFW via LHR",
totalPriceUsd: 1316,
displayPriceUsd: "$1,316 total ($658 pp x 2)",
notes: [
"Cheapest valid paired return captured in the fresh KAYAK sweep.",
"Skyscanner did not reproduce the exact low Jun 8 return cleanly.",
"Expedia hit a bot challenge on the Jun 8 paired-return check."
]
},
{
id: "quote-3",
source: "kayak",
legId: "return-solo",
passengerGroupIds: ["solo"],
bookingLink: "https://example.com/quote-3",
itinerarySummary: "BLQ -> DFW via MAD",
totalPriceUsd: 722,
displayPriceUsd: "$722 total",
notes: [
"Lowest valid solo-return fare in the fresh Jul window capture.",
"Skyscanner TH/USD showed the same Madrid connection at about $742.",
"Expedia surfaced a cheaper Turkish option, but it was excluded."
]
}
];
payload.rankedOptions = [
{
id: "primary",
title: "Best overall itinerary",
quoteIds: ["quote-1", "quote-2", "quote-3"],
totalPriceUsd: 4420,
rationale: "Best price-to-convenience tradeoff."
},
{
id: "backup",
title: "Backup itinerary",
quoteIds: ["quote-1", "quote-2", "quote-3"],
totalPriceUsd: 4515,
rationale: "Slightly higher price but still viable."
}
];
payload.executiveSummary = [
"Fresh Thailand-market sweeps shifted the outbound slightly earlier.",
"Lowest valid total observed was $4,420.",
"Cross-source corroboration is strongest for the outbound and solo return."
];
await renderFlightReportPdf(payload, outputPath);
const pdfContents = fs.readFileSync(outputPath, "latin1");
assert.doesNotMatch(pdfContents, /\/Users\/stefano\/\.openclaw\/workspace\/reports\//);
assert.ok(countPdfPages(outputPath) <= 4);
});
test("renderFlightReportPdf rejects incomplete report payloads", async () => {
const outputPath = path.join(os.tmpdir(), `flight-finder-incomplete-${Date.now()}.pdf`);
await assert.rejects(
() =>
renderFlightReportPdf(
{
...samplePayload(),
rankedOptions: [],
executiveSummary: []
},
outputPath
),
ReportValidationError
);
});
test("renderFlightReportPdf rejects payloads that are not marked as fresh-search output", async () => {
const outputPath = path.join(os.tmpdir(), `flight-finder-stale-${Date.now()}.pdf`);
await assert.rejects(
() =>
renderFlightReportPdf(
{
...samplePayload(),
searchExecution: {
...samplePayload().searchExecution,
freshSearch: false
}
},
outputPath
),
ReportValidationError
);
});
test("describeQuoteBookingLinks includes both aggregator and airline-direct links when captured", () => {
const payload = samplePayload();
const quote = {
...payload.quotes[0],
airlineName: "British Airways",
directBookingUrl: "https://www.britishairways.com/"
};
assert.deepEqual(describeQuoteBookingLinks(quote), [
{
label: "Search / book this fare",
text: "Search / book this fare: $2,847 via kayak",
url: "https://example.com/quote-1"
},
{
label: "Direct airline booking",
text: "Direct airline booking: British Airways",
url: "https://www.britishairways.com/"
}
]);
});
test("describeQuoteBookingLinks calls out missing direct-airline links explicitly", () => {
const payload = samplePayload();
assert.deepEqual(describeQuoteBookingLinks(payload.quotes[0]), [
{
label: "Search / book this fare",
text: "Search / book this fare: $2,847 via kayak",
url: "https://example.com/quote-1"
},
{
label: "Direct airline booking",
text: "Direct airline booking: not captured in this run",
url: null
}
]);
});
test("formatArtifactsRoot only shortens known workspace report paths", () => {
assert.equal(
formatArtifactsRoot("/Users/stefano/.openclaw/workspace/reports/dfw-blq-flight-report-2026-03-30-fresh-185742"),
"reports/dfw-blq-flight-report-2026-03-30-fresh-185742"
);
assert.equal(formatArtifactsRoot("/tmp/run-123"), "run-123");
});
@@ -0,0 +1,106 @@
import test from "node:test";
import assert from "node:assert/strict";
import {
DFW_BLQ_2026_PROMPT_DRAFT,
normalizeFlightReportRequest
} from "../src/request-normalizer.js";
import { getFlightReportStatus } from "../src/report-status.js";
import type { FlightReportPayload } from "../src/types.js";
function buildPayload(): FlightReportPayload {
return {
request: normalizeFlightReportRequest(DFW_BLQ_2026_PROMPT_DRAFT).request,
searchExecution: {
freshSearch: true,
startedAt: "2026-03-30T20:45:00Z",
completedAt: "2026-03-30T21:00:00Z",
artifactsRoot: "/tmp/flight-finder-report-status",
notes: ["Fresh bounded search executed for this report."]
},
sourceFindings: [
{
source: "kayak",
status: "viable",
checkedAt: "2026-03-30T21:00:00Z",
notes: ["Returned usable options."]
}
],
quotes: [
{
id: "quote-1",
source: "kayak",
legId: "outbound",
passengerGroupIds: ["pair", "solo"],
bookingLink: "https://example.com/quote-1",
itinerarySummary: "DFW -> BLQ via LHR",
totalPriceUsd: 2847,
displayPriceUsd: "$2,847",
crossCheckStatus: "not-available"
}
],
rankedOptions: [
{
id: "primary",
title: "Best overall itinerary",
quoteIds: ["quote-1"],
totalPriceUsd: 2847,
rationale: "Best price-to-convenience tradeoff."
}
],
executiveSummary: ["Best fare currently found is $2,847 total for 3 adults."],
reportWarnings: [],
degradedReasons: [],
comparisonCurrency: "USD",
generatedAt: "2026-03-30T21:00:00Z",
lastCompletedPhase: "ranking"
};
}
test("getFlightReportStatus treats missing recipient email as delivery-only blocker", () => {
const normalized = normalizeFlightReportRequest({
...DFW_BLQ_2026_PROMPT_DRAFT,
recipientEmail: null
});
const payload = {
...buildPayload(),
request: normalized.request
};
const status = getFlightReportStatus(normalized, payload);
assert.equal(status.readyToSearch, true);
assert.equal(status.pdfReady, true);
assert.equal(status.emailReady, false);
assert.deepEqual(status.needsMissingInputs, ["recipient email"]);
});
test("getFlightReportStatus marks all-sources-failed as a blocked but chat-summarizable outcome", () => {
const normalized = normalizeFlightReportRequest(DFW_BLQ_2026_PROMPT_DRAFT);
const status = getFlightReportStatus(normalized, {
...buildPayload(),
sourceFindings: [
{
source: "kayak",
status: "blocked",
checkedAt: "2026-03-30T21:00:00Z",
notes: ["Timed out."]
},
{
source: "skyscanner",
status: "blocked",
checkedAt: "2026-03-30T21:00:00Z",
notes: ["Timed out."]
}
],
quotes: [],
rankedOptions: [],
executiveSummary: [],
degradedReasons: ["All configured travel sources failed."]
});
assert.equal(status.terminalOutcome, "all-sources-failed");
assert.equal(status.chatSummaryReady, true);
assert.equal(status.pdfReady, false);
assert.equal(status.degraded, true);
});
@@ -0,0 +1,98 @@
import test from "node:test";
import assert from "node:assert/strict";
import {
DFW_BLQ_2026_PROMPT_DRAFT,
normalizeFlightReportRequest
} from "../src/request-normalizer.js";
test("normalizeFlightReportRequest preserves the legacy DFW-BLQ split-return structure", () => {
const result = normalizeFlightReportRequest(DFW_BLQ_2026_PROMPT_DRAFT);
assert.equal(result.readyToSearch, true);
assert.deepEqual(result.missingSearchInputs, []);
assert.deepEqual(result.missingDeliveryInputs, []);
assert.equal(result.request.preferences.marketCountry, "TH");
assert.equal(result.request.preferences.normalizeCurrencyTo, "USD");
assert.equal(result.request.legs.length, 3);
assert.equal(result.request.passengerGroups.length, 2);
assert.deepEqual(
result.request.legAssignments.find((entry) => entry.legId === "outbound")?.passengerGroupIds,
["pair", "solo"]
);
assert.equal(
result.request.preferences.requireAirlineDirectCrossCheck,
true
);
});
test("normalizeFlightReportRequest keeps email missing as a delivery gate, not a search blocker", () => {
const result = normalizeFlightReportRequest({
...DFW_BLQ_2026_PROMPT_DRAFT,
recipientEmail: null
});
assert.equal(result.readyToSearch, true);
assert.deepEqual(result.missingSearchInputs, []);
assert.deepEqual(result.missingDeliveryInputs, ["recipient email"]);
assert.match(result.warnings.join(" "), /recipient email/i);
});
test("normalizeFlightReportRequest reports missing search-critical inputs explicitly", () => {
const result = normalizeFlightReportRequest({
tripName: "Incomplete flight request",
legs: [
{
id: "outbound",
origin: "DFW",
destination: "",
label: "Outbound"
}
],
passengerGroups: [],
legAssignments: []
});
assert.equal(result.readyToSearch, false);
assert.match(result.missingSearchInputs.join(" | "), /origin and destination/i);
assert.match(result.missingSearchInputs.join(" | "), /date window/i);
assert.match(result.missingSearchInputs.join(" | "), /passenger groups/i);
assert.match(result.missingSearchInputs.join(" | "), /assignments/i);
});
test("normalizeFlightReportRequest validates marketCountry as an ISO alpha-2 code", () => {
assert.throws(
() =>
normalizeFlightReportRequest({
...DFW_BLQ_2026_PROMPT_DRAFT,
preferences: {
...DFW_BLQ_2026_PROMPT_DRAFT.preferences,
marketCountry: "Germany"
}
}),
/ISO 3166-1 alpha-2/i
);
});
test("normalizeFlightReportRequest uppercases a lowercase marketCountry code", () => {
const result = normalizeFlightReportRequest({
...DFW_BLQ_2026_PROMPT_DRAFT,
preferences: {
...DFW_BLQ_2026_PROMPT_DRAFT.preferences,
marketCountry: "th"
}
});
assert.equal(result.request.preferences.marketCountry, "TH");
});
test("normalizeFlightReportRequest flags a malformed recipient email without blocking search readiness", () => {
const result = normalizeFlightReportRequest({
...DFW_BLQ_2026_PROMPT_DRAFT,
recipientEmail: "stefano-at-example"
});
assert.equal(result.readyToSearch, true);
assert.deepEqual(result.missingDeliveryInputs, ["valid recipient email"]);
assert.match(result.warnings.join(" "), /malformed/i);
});
@@ -0,0 +1,42 @@
import test from "node:test";
import assert from "node:assert/strict";
import os from "node:os";
import path from "node:path";
import fs from "node:fs/promises";
import {
clearFlightFinderRunState,
loadFlightFinderRunState,
saveFlightFinderRunState
} from "../src/run-state.js";
import {
DFW_BLQ_2026_PROMPT_DRAFT,
normalizeFlightReportRequest
} from "../src/request-normalizer.js";
test("save/load/clearFlightFinderRunState persists the resumable phase context", async () => {
const baseDir = await fs.mkdtemp(
path.join(os.tmpdir(), "flight-finder-run-state-")
);
const request = normalizeFlightReportRequest(DFW_BLQ_2026_PROMPT_DRAFT).request;
await saveFlightFinderRunState(
{
request,
lastCompletedPhase: "search",
updatedAt: "2026-03-30T21:00:00Z",
reportWarnings: [],
degradedReasons: [],
sourceFindings: [],
comparisonCurrency: "USD"
},
baseDir
);
const loaded = await loadFlightFinderRunState(baseDir);
assert.equal(loaded?.lastCompletedPhase, "search");
assert.equal(loaded?.request.tripName, "DFW ↔ BLQ flight report");
await clearFlightFinderRunState(baseDir);
assert.equal(await loadFlightFinderRunState(baseDir), null);
});
@@ -0,0 +1,46 @@
import test from "node:test";
import assert from "node:assert/strict";
import {
DFW_BLQ_2026_PROMPT_DRAFT,
normalizeFlightReportRequest
} from "../src/request-normalizer.js";
import {
VPN_CONNECT_TIMEOUT_MS,
VPN_DISCONNECT_TIMEOUT_MS,
buildFlightSearchPlan
} from "../src/search-orchestration.js";
test("buildFlightSearchPlan activates VPN only when marketCountry is explicit", () => {
const request = normalizeFlightReportRequest(DFW_BLQ_2026_PROMPT_DRAFT).request;
const plan = buildFlightSearchPlan(request);
assert.equal(plan.vpn.enabled, true);
assert.equal(plan.vpn.marketCountry, "TH");
assert.equal(plan.vpn.connectTimeoutMs, VPN_CONNECT_TIMEOUT_MS);
assert.equal(plan.vpn.disconnectTimeoutMs, VPN_DISCONNECT_TIMEOUT_MS);
});
test("buildFlightSearchPlan keeps airline-direct best-effort when degraded", () => {
const request = normalizeFlightReportRequest({
...DFW_BLQ_2026_PROMPT_DRAFT,
preferences: {
...DFW_BLQ_2026_PROMPT_DRAFT.preferences,
marketCountry: null
}
}).request;
const plan = buildFlightSearchPlan(request, [
{
source: "airline-direct",
status: "degraded",
checkedAt: "2026-03-30T21:00:00Z",
notes: ["Direct booking shell loads but search completion is unreliable."]
}
]);
assert.equal(plan.vpn.enabled, false);
const directSource = plan.sourceOrder.find((entry) => entry.source === "airline-direct");
assert.equal(directSource?.enabled, true);
assert.equal(directSource?.required, false);
assert.match(plan.degradedReasons.join(" "), /airline-direct/i);
});
+17
View File
@@ -0,0 +1,17 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "ESNext",
"moduleResolution": "bundler",
"esModuleInterop": true,
"allowSyntheticDefaultImports": true,
"strict": true,
"skipLibCheck": true,
"resolveJsonModule": true,
"types": ["node"],
"outDir": "./dist",
"rootDir": "."
},
"include": ["src/**/*.ts", "tests/**/*.ts"],
"exclude": ["node_modules", "dist"]
}
+120
View File
@@ -0,0 +1,120 @@
---
name: nordvpn-client
description: Use when managing NordVPN on macOS or Linux, including install/bootstrap, login, connect, disconnect, status checks, or verifying a VPN location before running another skill.
---
# NordVPN Client
Cross-platform NordVPN lifecycle management for macOS and Linux hosts.
## Use This Skill For
- probing whether NordVPN automation is ready
- bootstrapping missing backend dependencies
- validating auth
- connecting to a country or city
- verifying the public exit location
- disconnecting and restoring the normal network state
## Command Surface
```bash
node scripts/nordvpn-client.js status
node scripts/nordvpn-client.js install
node scripts/nordvpn-client.js login
node scripts/nordvpn-client.js verify
node scripts/nordvpn-client.js verify --country "Germany"
node scripts/nordvpn-client.js verify --country "Japan" --city "Tokyo"
node scripts/nordvpn-client.js connect --country "Germany"
node scripts/nordvpn-client.js connect --country "Japan" --city "Tokyo"
node scripts/nordvpn-client.js disconnect
node scripts/nordvpn-client.js status --debug
```
## Backend Model
- Linux:
- use the official `nordvpn` CLI
- `install` uses the official NordVPN installer
- token login is supported
- macOS:
- use NordLynx/WireGuard through `wireguard-go` and `wireguard-tools`
- `install` bootstraps them with Homebrew
- `login` validates the token for the WireGuard backend
- the generated WireGuard config stays free of `DNS = ...`
- `connect` now requires a bounded persistence gate plus a verified exit before success is declared
- the skill snapshots and applies NordVPN DNS only to eligible physical services while connected
- NordVPN DNS is applied only after the tunnel remains up, the final liveness check still shows the requested exit, and system hostname resolution still works afterward
- `disconnect` restores the saved DNS/search-domain state even if the tunnel state is stale
- Tailscale is suspended before connect and resumed after disconnect or failed connect
- the skill writes a short-lived Tailscale suppression marker during VPN connect so host watchdogs do not immediately re-run `tailscale up`
- `NordVPN.app` may remain installed but is only the manual fallback
## Credentials
Default OpenClaw credential paths:
- token: `~/.openclaw/workspace/.clawdbot/credentials/nordvpn/token.txt`
- password: `~/.openclaw/workspace/.clawdbot/credentials/nordvpn/password.txt`
Supported env vars:
- `NORDVPN_TOKEN`
- `NORDVPN_TOKEN_FILE`
- `NORDVPN_USERNAME`
- `NORDVPN_PASSWORD`
- `NORDVPN_PASSWORD_FILE`
## macOS Requirements
Automated macOS connects require all of:
- `wireguard-go`
- `wireguard-tools`
- `NORDVPN_TOKEN` or the default token file
- non-interactive `sudo` for the installed helper script:
- `~/.openclaw/workspace/skills/nordvpn-client/scripts/nordvpn-wireguard-helper.sh`
Exact `visudo` rule for the installed OpenClaw skill:
```sudoers
stefano ALL=(root) NOPASSWD: /Users/stefano/.openclaw/workspace/skills/nordvpn-client/scripts/nordvpn-wireguard-helper.sh probe, /Users/stefano/.openclaw/workspace/skills/nordvpn-client/scripts/nordvpn-wireguard-helper.sh up, /Users/stefano/.openclaw/workspace/skills/nordvpn-client/scripts/nordvpn-wireguard-helper.sh down
```
Operational note:
- the persistence gate reuses the already-allowed `probe` action to confirm the live `utun*` WireGuard runtime and does not require extra sudoers actions beyond `probe`, `up`, and `down`
## Agent Guidance
- run `status` first when the machine state is unclear
- on macOS, if tooling is missing, run `install`
- if auth is unclear, run `login`
- use `connect` before location-sensitive skills such as `web-automation`
- use `verify` after connect when you need an explicit location check
- use `disconnect` after the follow-up task
- if `connect` fails its persistence or final verification gate, treat that as a safe rollback, not a partial success
## Output Rules
- normal JSON output redacts local path metadata and helper-hardening diagnostics
- use `--debug` only when deeper troubleshooting requires internal local paths and helper/config metadata
## Troubleshooting Cues
- `Invalid authorization header`:
- token file exists but the token is invalid; replace the token and rerun `login`
- `sudoReady: false`:
- the helper is not allowed in sudoers; add the `visudo` rule above
- connect succeeds but final state looks inconsistent:
- rely on the verified public IP/location first
- then inspect `status --debug`
- `verified: true` but `persistence.stable: false` should not happen anymore; if it does, the skill should roll back instead of pinning DNS
- disconnect should leave:
- normal public IP restored
- no active WireGuard state
- Tailscale resumed if the skill suspended it
For full operator setup and troubleshooting, see:
- `docs/nordvpn-client.md`
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,671 @@
const test = require("node:test");
const assert = require("node:assert/strict");
const fs = require("node:fs");
const path = require("node:path");
const vm = require("node:vm");
function loadInternals() {
const scriptPath = path.join(__dirname, "nordvpn-client.js");
const source = fs.readFileSync(scriptPath, "utf8").replace(/\nmain\(\);\s*$/, "\n");
const wrapped = `${source}
module.exports = {
buildMacTailscaleState:
typeof buildMacTailscaleState === "function" ? buildMacTailscaleState : undefined,
markMacTailscaleRecoverySuppressed:
typeof markMacTailscaleRecoverySuppressed === "function" ? markMacTailscaleRecoverySuppressed : undefined,
clearMacTailscaleRecoverySuppressed:
typeof clearMacTailscaleRecoverySuppressed === "function" ? clearMacTailscaleRecoverySuppressed : undefined,
buildMacDnsState:
typeof buildMacDnsState === "function" ? buildMacDnsState : undefined,
buildWireguardConfig:
typeof buildWireguardConfig === "function" ? buildWireguardConfig : undefined,
buildLookupResult:
typeof buildLookupResult === "function" ? buildLookupResult : undefined,
cleanupMacWireguardState:
typeof cleanupMacWireguardState === "function" ? cleanupMacWireguardState : undefined,
cleanupMacWireguardAndDnsState:
typeof cleanupMacWireguardAndDnsState === "function" ? cleanupMacWireguardAndDnsState : undefined,
collectMacWireguardDiagnostics:
typeof collectMacWireguardDiagnostics === "function" ? collectMacWireguardDiagnostics : undefined,
acquireOperationLock:
typeof acquireOperationLock === "function" ? acquireOperationLock : undefined,
inspectMacWireguardHelperSecurity:
typeof inspectMacWireguardHelperSecurity === "function" ? inspectMacWireguardHelperSecurity : undefined,
getMacTailscalePath:
typeof getMacTailscalePath === "function" ? getMacTailscalePath : undefined,
isBenignMacWireguardAbsentError:
typeof isBenignMacWireguardAbsentError === "function" ? isBenignMacWireguardAbsentError : undefined,
isMacTailscaleActive:
typeof isMacTailscaleActive === "function" ? isMacTailscaleActive : undefined,
checkMacWireguardPersistence:
typeof checkMacWireguardPersistence === "function" ? checkMacWireguardPersistence : undefined,
normalizeSuccessfulConnectState:
typeof normalizeSuccessfulConnectState === "function" ? normalizeSuccessfulConnectState : undefined,
normalizeStatusState:
typeof normalizeStatusState === "function" ? normalizeStatusState : undefined,
parseMacWireguardHelperStatus:
typeof parseMacWireguardHelperStatus === "function" ? parseMacWireguardHelperStatus : undefined,
shouldRejectMacDnsBaseline:
typeof shouldRejectMacDnsBaseline === "function" ? shouldRejectMacDnsBaseline : undefined,
shouldManageMacDnsService:
typeof shouldManageMacDnsService === "function" ? shouldManageMacDnsService : undefined,
sanitizeOutputPayload:
typeof sanitizeOutputPayload === "function" ? sanitizeOutputPayload : undefined,
shouldFinalizeMacWireguardConnect:
typeof shouldFinalizeMacWireguardConnect === "function" ? shouldFinalizeMacWireguardConnect : undefined,
shouldResumeMacTailscale:
typeof shouldResumeMacTailscale === "function" ? shouldResumeMacTailscale : undefined,
shouldAttemptMacWireguardDisconnect:
typeof shouldAttemptMacWireguardDisconnect === "function" ? shouldAttemptMacWireguardDisconnect : undefined,
detectMacWireguardActiveFromIfconfig:
typeof detectMacWireguardActiveFromIfconfig === "function" ? detectMacWireguardActiveFromIfconfig : undefined,
resolveHostnameWithFallback:
typeof resolveHostnameWithFallback === "function" ? resolveHostnameWithFallback : undefined,
verifySystemHostnameResolution:
typeof verifySystemHostnameResolution === "function" ? verifySystemHostnameResolution : undefined,
verifyConnectionWithRetry:
typeof verifyConnectionWithRetry === "function" ? verifyConnectionWithRetry : undefined,
};`;
const sandbox = {
require,
module: { exports: {} },
exports: {},
__dirname,
__filename: scriptPath,
process: { ...process, exit() {} },
console,
setTimeout,
clearTimeout,
Buffer,
};
vm.runInNewContext(wrapped, sandbox, { filename: scriptPath });
return sandbox.module.exports;
}
test("detectMacWireguardActiveFromIfconfig detects nordvpn utun client address", () => {
const { detectMacWireguardActiveFromIfconfig } = loadInternals();
assert.equal(typeof detectMacWireguardActiveFromIfconfig, "function");
const ifconfig = `
utun8: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1380
utun9: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1420
\tinet 10.5.0.2 --> 10.5.0.2 netmask 0xff000000
`;
assert.equal(detectMacWireguardActiveFromIfconfig(ifconfig), true);
assert.equal(detectMacWireguardActiveFromIfconfig("utun7: flags=8051\n\tinet 100.64.0.4"), false);
});
test("buildLookupResult supports lookup all=true mode", () => {
const { buildLookupResult } = loadInternals();
assert.equal(typeof buildLookupResult, "function");
assert.equal(
JSON.stringify(buildLookupResult("104.26.9.44", { all: true })),
JSON.stringify([{ address: "104.26.9.44", family: 4 }])
);
assert.equal(JSON.stringify(buildLookupResult("104.26.9.44", { all: false })), JSON.stringify(["104.26.9.44", 4]));
});
test("buildWireguardConfig omits DNS and relies on post-connect networksetup on macOS", () => {
const { buildWireguardConfig } = loadInternals();
assert.equal(typeof buildWireguardConfig, "function");
const config = buildWireguardConfig(
{
hostname: "tr73.nordvpn.com",
ips: [{ ip: { version: 4, ip: "45.89.52.1" } }],
technologies: [{ identifier: "wireguard_udp", metadata: [{ name: "public_key", value: "PUBKEY" }] }],
},
"PRIVATEKEY"
);
assert.equal(config.includes("DNS = 103.86.96.100, 103.86.99.100"), false);
assert.equal(config.includes("AllowedIPs = 0.0.0.0/0"), true);
});
test("shouldManageMacDnsService keeps active physical services and excludes virtual ones", () => {
const { shouldManageMacDnsService } = loadInternals();
assert.equal(typeof shouldManageMacDnsService, "function");
assert.equal(shouldManageMacDnsService("Wi-Fi"), true);
assert.equal(shouldManageMacDnsService("USB 10/100/1000 LAN"), true);
assert.equal(shouldManageMacDnsService("Tailscale"), false);
assert.equal(shouldManageMacDnsService("Thunderbolt Bridge"), false);
assert.equal(shouldManageMacDnsService("Acme VPN"), false);
});
test("buildMacDnsState records DNS and search domains per service", () => {
const { buildMacDnsState } = loadInternals();
assert.equal(typeof buildMacDnsState, "function");
assert.equal(
JSON.stringify(
buildMacDnsState([
{ name: "Wi-Fi", dnsServers: ["1.1.1.1"], searchDomains: [] },
{ name: "USB 10/100/1000 LAN", dnsServers: [], searchDomains: ["lan"] },
])
),
JSON.stringify({
services: [
{ name: "Wi-Fi", dnsServers: ["1.1.1.1"], searchDomains: [] },
{ name: "USB 10/100/1000 LAN", dnsServers: [], searchDomains: ["lan"] },
],
})
);
});
test("shouldRejectMacDnsBaseline flags a NordVPN-pinned restore snapshot", () => {
const { shouldRejectMacDnsBaseline } = loadInternals();
assert.equal(typeof shouldRejectMacDnsBaseline, "function");
assert.equal(
shouldRejectMacDnsBaseline({
services: [
{
name: "Wi-Fi",
dnsServers: ["103.86.96.100", "103.86.99.100"],
searchDomains: [],
},
],
}),
true
);
assert.equal(
shouldRejectMacDnsBaseline({
services: [
{
name: "Wi-Fi",
dnsServers: ["1.1.1.1"],
searchDomains: [],
},
],
}),
false
);
});
test("getMacTailscalePath falls back to /opt/homebrew/bin/tailscale when PATH lookup is missing", () => {
const { getMacTailscalePath } = loadInternals();
assert.equal(typeof getMacTailscalePath, "function");
assert.equal(
getMacTailscalePath({
commandExists: () => "",
fileExists: (target) => target === "/opt/homebrew/bin/tailscale",
}),
"/opt/homebrew/bin/tailscale"
);
});
test("buildMacTailscaleState records whether tailscale was active", () => {
const { buildMacTailscaleState } = loadInternals();
assert.equal(typeof buildMacTailscaleState, "function");
assert.equal(
JSON.stringify(buildMacTailscaleState(true)),
JSON.stringify({
tailscaleWasActive: true,
})
);
});
test("tailscale recovery suppression marker can be created and cleared", () => {
const { markMacTailscaleRecoverySuppressed, clearMacTailscaleRecoverySuppressed } = loadInternals();
assert.equal(typeof markMacTailscaleRecoverySuppressed, "function");
assert.equal(typeof clearMacTailscaleRecoverySuppressed, "function");
const markerPath = path.join(process.env.HOME || "", ".nordvpn-client", "tailscale-suppressed");
clearMacTailscaleRecoverySuppressed();
markMacTailscaleRecoverySuppressed();
assert.equal(fs.existsSync(markerPath), true);
clearMacTailscaleRecoverySuppressed();
assert.equal(fs.existsSync(markerPath), false);
});
test("shouldResumeMacTailscale only resumes when previously active and not already running", () => {
const { shouldResumeMacTailscale } = loadInternals();
assert.equal(typeof shouldResumeMacTailscale, "function");
assert.equal(shouldResumeMacTailscale({ tailscaleWasActive: true }, false), true);
assert.equal(shouldResumeMacTailscale({ tailscaleWasActive: true }, true), false);
assert.equal(shouldResumeMacTailscale({ tailscaleWasActive: false }, false), false);
assert.equal(shouldResumeMacTailscale(null, false), false);
});
test("cleanupMacWireguardState removes stale config and last-connection files", () => {
const { cleanupMacWireguardState } = loadInternals();
assert.equal(typeof cleanupMacWireguardState, "function");
const tmpDir = fs.mkdtempSync(path.join(fs.mkdtempSync("/tmp/nordvpn-client-test-"), "state-"));
const configPath = path.join(tmpDir, "nordvpnctl.conf");
const lastConnectionPath = path.join(tmpDir, "last-connection.json");
fs.writeFileSync(configPath, "wireguard-config");
fs.writeFileSync(lastConnectionPath, "{\"country\":\"Germany\"}");
const result = cleanupMacWireguardState({
configPath,
lastConnectionPath,
});
assert.equal(result.cleaned, true);
assert.equal(fs.existsSync(configPath), false);
assert.equal(fs.existsSync(lastConnectionPath), false);
});
test("cleanupMacWireguardAndDnsState removes stale config, DNS snapshot, and last-connection files", () => {
const { cleanupMacWireguardAndDnsState } = loadInternals();
assert.equal(typeof cleanupMacWireguardAndDnsState, "function");
const tmpDir = fs.mkdtempSync(path.join(fs.mkdtempSync("/tmp/nordvpn-client-test-"), "state-"));
const configPath = path.join(tmpDir, "nordvpnctl.conf");
const lastConnectionPath = path.join(tmpDir, "last-connection.json");
const dnsStatePath = path.join(tmpDir, "dns.json");
fs.writeFileSync(configPath, "wireguard-config");
fs.writeFileSync(lastConnectionPath, "{\"country\":\"Germany\"}");
fs.writeFileSync(dnsStatePath, "{\"services\":[]}");
const result = cleanupMacWireguardAndDnsState({
configPath,
lastConnectionPath,
dnsStatePath,
});
assert.equal(result.cleaned, true);
assert.equal(fs.existsSync(configPath), false);
assert.equal(fs.existsSync(lastConnectionPath), false);
assert.equal(fs.existsSync(dnsStatePath), false);
});
test("inspectMacWireguardHelperSecurity rejects a user-owned helper path", () => {
const { inspectMacWireguardHelperSecurity } = loadInternals();
assert.equal(typeof inspectMacWireguardHelperSecurity, "function");
const result = inspectMacWireguardHelperSecurity("/tmp/nordvpn-wireguard-helper.sh", {
fileExists: () => true,
statSync: () => ({
uid: 501,
gid: 20,
mode: 0o100755,
}),
});
assert.equal(result.exists, true);
assert.equal(result.hardened, false);
assert.match(result.reason, /root-owned/i);
});
test("inspectMacWireguardHelperSecurity accepts a root-owned non-writable helper path", () => {
const { inspectMacWireguardHelperSecurity } = loadInternals();
assert.equal(typeof inspectMacWireguardHelperSecurity, "function");
const result = inspectMacWireguardHelperSecurity("/tmp/nordvpn-wireguard-helper.sh", {
fileExists: () => true,
statSync: () => ({
uid: 0,
gid: 0,
mode: 0o100755,
}),
});
assert.equal(result.exists, true);
assert.equal(result.hardened, true);
assert.equal(result.reason, "");
});
test("collectMacWireguardDiagnostics captures bounded wg/ifconfig/route/process output", async () => {
const { collectMacWireguardDiagnostics } = loadInternals();
assert.equal(typeof collectMacWireguardDiagnostics, "function");
const seen = [];
const result = await collectMacWireguardDiagnostics({
interfaceName: "nordvpnctl",
runExec: async (command, args) => {
seen.push(`${command} ${args.join(" ")}`);
if (command === "/opt/homebrew/bin/wg") {
return { ok: true, stdout: "interface: nordvpnctl\npeer: abc123", stderr: "", error: "" };
}
if (command === "ifconfig") {
return { ok: true, stdout: "utun8: flags=8051\n\tinet 10.5.0.2 --> 10.5.0.2", stderr: "", error: "" };
}
if (command === "route") {
return { ok: true, stdout: "default 10.5.0.2 UGSc", stderr: "", error: "" };
}
if (command === "pgrep") {
return { ok: true, stdout: "1234 wireguard-go utun\n5678 wg-quick up nordvpnctl", stderr: "", error: "" };
}
throw new Error(`unexpected command: ${command}`);
},
});
assert.deepEqual(seen, [
"/opt/homebrew/bin/wg show nordvpnctl",
"ifconfig nordvpnctl",
"route -n get default",
"pgrep -fl wireguard-go|wg-quick|nordvpnctl",
]);
assert.equal(result.interfaceName, "nordvpnctl");
assert.equal(result.wgShow.includes("peer: abc123"), true);
assert.equal(result.ifconfig.includes("10.5.0.2"), true);
assert.equal(result.routes.includes("default 10.5.0.2"), true);
assert.equal(result.processes.includes("wireguard-go"), true);
});
test("parseMacWireguardHelperStatus reads active helper key-value output", () => {
const { parseMacWireguardHelperStatus } = loadInternals();
assert.equal(typeof parseMacWireguardHelperStatus, "function");
const result = parseMacWireguardHelperStatus("active=1\ninterfaceName=nordvpnctl\n");
assert.equal(result.active, true);
assert.equal(result.interfaceName, "nordvpnctl");
});
test("checkMacWireguardPersistence waits for both helper-active and verified exit", async () => {
const { checkMacWireguardPersistence } = loadInternals();
assert.equal(typeof checkMacWireguardPersistence, "function");
const helperStatuses = [
{ active: false, interfaceName: "nordvpnctl" },
{ active: true, interfaceName: "nordvpnctl" },
];
const verifications = [
{ ok: false, ipInfo: { ok: false, error: "timeout" } },
{ ok: true, ipInfo: { ok: true, country: "Germany", city: "Frankfurt" } },
];
const result = await checkMacWireguardPersistence(
{ country: "Germany", city: "" },
{
attempts: 2,
delayMs: 1,
getHelperStatus: async () => helperStatuses.shift(),
verifyConnection: async () => verifications.shift(),
sleep: async () => {},
}
);
assert.equal(result.stable, true);
assert.equal(result.attempts, 2);
assert.equal(result.helperStatus.active, true);
assert.equal(result.verified.ok, true);
});
test("checkMacWireguardPersistence returns the last failed status when stability is not reached", async () => {
const { checkMacWireguardPersistence } = loadInternals();
assert.equal(typeof checkMacWireguardPersistence, "function");
const result = await checkMacWireguardPersistence(
{ country: "Germany", city: "" },
{
attempts: 2,
delayMs: 1,
getHelperStatus: async () => ({ active: false, interfaceName: "nordvpnctl" }),
verifyConnection: async () => ({ ok: false, ipInfo: { ok: false, error: "timeout" } }),
sleep: async () => {},
}
);
assert.equal(result.stable, false);
assert.equal(result.attempts, 2);
assert.equal(result.helperStatus.active, false);
assert.equal(result.verified.ok, false);
});
test("acquireOperationLock cleans a stale dead-pid lock before taking ownership", () => {
const { acquireOperationLock } = loadInternals();
assert.equal(typeof acquireOperationLock, "function");
const tmpDir = fs.mkdtempSync(path.join(fs.mkdtempSync("/tmp/nordvpn-client-test-"), "lock-"));
const lockPath = path.join(tmpDir, "operation.lock");
fs.writeFileSync(
lockPath,
JSON.stringify({
action: "connect",
pid: 0,
startedAt: new Date(0).toISOString(),
startedAtMs: 0,
})
);
const lock = acquireOperationLock("disconnect", { lockPath });
const lockFile = JSON.parse(fs.readFileSync(lockPath, "utf8"));
assert.equal(lockFile.action, "disconnect");
assert.equal(lockFile.pid, process.pid);
lock.release();
assert.equal(fs.existsSync(lockPath), false);
});
test("shouldAttemptMacWireguardDisconnect does not trust active=false when residual state exists", () => {
const { shouldAttemptMacWireguardDisconnect } = loadInternals();
assert.equal(typeof shouldAttemptMacWireguardDisconnect, "function");
assert.equal(
shouldAttemptMacWireguardDisconnect({
active: false,
configPath: "/Users/stefano/.nordvpn-client/wireguard/nordvpnctl.conf",
endpoint: null,
lastConnection: null,
}),
true
);
assert.equal(
shouldAttemptMacWireguardDisconnect({
active: false,
configPath: null,
endpoint: null,
lastConnection: { country: "Italy" },
}),
true
);
assert.equal(
shouldAttemptMacWireguardDisconnect({
active: false,
configPath: null,
endpoint: null,
lastConnection: null,
}),
false
);
});
test("isBenignMacWireguardAbsentError recognizes stale-interface teardown errors", () => {
const { isBenignMacWireguardAbsentError } = loadInternals();
assert.equal(typeof isBenignMacWireguardAbsentError, "function");
assert.equal(isBenignMacWireguardAbsentError("wg-quick: `nordvpnctl' is not a WireGuard interface"), true);
assert.equal(isBenignMacWireguardAbsentError("Unable to access interface: No such file or directory"), true);
assert.equal(isBenignMacWireguardAbsentError("permission denied"), false);
});
test("normalizeSuccessfulConnectState marks the connect snapshot active after verified macOS wireguard connect", () => {
const { normalizeSuccessfulConnectState } = loadInternals();
assert.equal(typeof normalizeSuccessfulConnectState, "function");
const state = normalizeSuccessfulConnectState(
{
platform: "darwin",
controlMode: "wireguard",
connected: false,
wireguard: {
active: false,
endpoint: null,
},
},
{
backend: "wireguard",
server: {
hostname: "de1227.nordvpn.com",
},
},
{
ok: true,
ipInfo: {
country: "Germany",
},
}
);
assert.equal(state.connected, true);
assert.equal(state.wireguard.active, true);
assert.equal(state.wireguard.endpoint, "de1227.nordvpn.com:51820");
});
test("shouldFinalizeMacWireguardConnect requires a verified and stable wireguard connect", () => {
const { shouldFinalizeMacWireguardConnect } = loadInternals();
assert.equal(typeof shouldFinalizeMacWireguardConnect, "function");
assert.equal(shouldFinalizeMacWireguardConnect({ backend: "wireguard" }, { ok: true }, { stable: true }), true);
assert.equal(shouldFinalizeMacWireguardConnect({ backend: "wireguard" }, { ok: true }, { stable: false }), false);
assert.equal(shouldFinalizeMacWireguardConnect({ backend: "wireguard" }, { ok: false }, { stable: true }), false);
assert.equal(shouldFinalizeMacWireguardConnect({ backend: "cli" }, { ok: true }, { stable: true }), false);
assert.equal(shouldFinalizeMacWireguardConnect(null, { ok: true }, { stable: true }), false);
});
test("normalizeStatusState marks macOS wireguard connected when public IP matches the last successful target", () => {
const { normalizeStatusState } = loadInternals();
assert.equal(typeof normalizeStatusState, "function");
const state = normalizeStatusState({
platform: "darwin",
controlMode: "wireguard",
connected: false,
wireguard: {
active: false,
endpoint: "tr73.nordvpn.com:51820",
lastConnection: {
requestedTarget: { country: "Turkey", city: "" },
resolvedTarget: { country: "Turkey", city: "Istanbul" },
},
},
publicIp: {
ok: true,
country: "Turkey",
city: "Istanbul",
},
});
assert.equal(state.connected, true);
assert.equal(state.wireguard.active, true);
});
test("sanitizeOutputPayload redacts local path metadata from normal JSON output", () => {
const { sanitizeOutputPayload } = loadInternals();
assert.equal(typeof sanitizeOutputPayload, "function");
const sanitized = sanitizeOutputPayload({
cliPath: "/opt/homebrew/bin/nordvpn",
appPath: "/Applications/NordVPN.app",
wireguard: {
configPath: "/Users/stefano/.nordvpn-client/wireguard/nordvpnctl.conf",
helperPath: "/Users/stefano/.openclaw/workspace/skills/nordvpn-client/scripts/nordvpn-wireguard-helper.sh",
helperSecurity: {
hardened: false,
reason: "Helper must be root-owned before privileged actions are trusted.",
},
authCache: {
tokenSource: "default:/Users/stefano/.openclaw/workspace/.clawdbot/credentials/nordvpn/token.txt",
},
endpoint: "jp454.nordvpn.com:51820",
},
});
assert.equal(sanitized.cliPath, null);
assert.equal(sanitized.appPath, null);
assert.equal(sanitized.wireguard.configPath, null);
assert.equal(sanitized.wireguard.helperPath, null);
assert.equal(sanitized.wireguard.helperSecurity, null);
assert.equal(sanitized.wireguard.authCache.tokenSource, null);
assert.equal(sanitized.wireguard.endpoint, "jp454.nordvpn.com:51820");
});
test("isMacTailscaleActive treats Running backend as active", () => {
const { isMacTailscaleActive } = loadInternals();
assert.equal(typeof isMacTailscaleActive, "function");
assert.equal(isMacTailscaleActive({ BackendState: "Running" }), true);
assert.equal(isMacTailscaleActive({ BackendState: "Stopped" }), false);
});
test("verifyConnectionWithRetry retries transient reachability failures", async () => {
const { verifyConnectionWithRetry } = loadInternals();
assert.equal(typeof verifyConnectionWithRetry, "function");
let attempts = 0;
const result = await verifyConnectionWithRetry(
{ country: "Italy", city: "Milan" },
{
attempts: 3,
delayMs: 1,
getPublicIpInfo: async () => {
attempts += 1;
if (attempts === 1) {
return { ok: false, error: "read EHOSTUNREACH" };
}
return { ok: true, country: "Italy", city: "Milan" };
},
}
);
assert.equal(result.ok, true);
assert.equal(result.ipInfo.country, "Italy");
assert.equal(attempts, 2);
});
test("resolveHostnameWithFallback uses fallback resolvers when system lookup fails", async () => {
const { resolveHostnameWithFallback } = loadInternals();
assert.equal(typeof resolveHostnameWithFallback, "function");
const calls = [];
const address = await resolveHostnameWithFallback("ipapi.co", {
resolvers: ["1.1.1.1", "8.8.8.8"],
resolveWithResolver: async (hostname, resolver) => {
calls.push(`${resolver}:${hostname}`);
if (resolver === "1.1.1.1") return [];
return ["104.26.9.44"];
},
});
assert.equal(address, "104.26.9.44");
assert.deepEqual(calls, ["1.1.1.1:ipapi.co", "8.8.8.8:ipapi.co"]);
});
test("verifySystemHostnameResolution succeeds when any system lookup resolves", async () => {
const { verifySystemHostnameResolution } = loadInternals();
assert.equal(typeof verifySystemHostnameResolution, "function");
const calls = [];
const result = await verifySystemHostnameResolution(["www.google.com", "api.openai.com"], {
timeoutMs: 5,
lookup: async (hostname) => {
calls.push(hostname);
if (hostname === "www.google.com") {
throw new Error("ENOTFOUND");
}
return { address: "104.18.33.45", family: 4 };
},
});
assert.equal(result.ok, true);
assert.equal(result.hostname, "api.openai.com");
assert.equal(result.address, "104.18.33.45");
assert.deepEqual(calls, ["www.google.com", "api.openai.com"]);
});
test("verifySystemHostnameResolution fails when all hostnames fail system lookup", async () => {
const { verifySystemHostnameResolution } = loadInternals();
assert.equal(typeof verifySystemHostnameResolution, "function");
const result = await verifySystemHostnameResolution(["www.google.com", "api.openai.com"], {
timeoutMs: 5,
lookup: async (hostname) => {
throw new Error(`${hostname}: timeout`);
},
});
assert.equal(result.ok, false);
assert.equal(result.hostname, "");
assert.match(result.error, /www\.google\.com/);
assert.match(result.error, /api\.openai\.com/);
});
+56
View File
@@ -0,0 +1,56 @@
#!/bin/sh
set -eu
ACTION="${1:-}"
case "$ACTION" in
probe|up|down|status|cleanup)
;;
*)
echo "Usage: nordvpn-wireguard-helper.sh [probe|up|down|status|cleanup]" >&2
exit 2
;;
esac
WG_QUICK="/opt/homebrew/bin/wg-quick"
WG="/opt/homebrew/bin/wg"
WG_CONFIG="/Users/stefano/.nordvpn-client/wireguard/nordvpnctl.conf"
WG_INTERFACE="nordvpnctl"
PATH="/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin"
export PATH
if [ "$ACTION" = "probe" ] || [ "$ACTION" = "status" ]; then
test -x "$WG_QUICK"
ACTIVE=0
RUNTIME_INTERFACE=""
if [ -x "$WG" ]; then
RUNTIME_INTERFACE=$("$WG" show interfaces 2>/dev/null | awk 'NF { print $1; exit }')
fi
if [ -n "$RUNTIME_INTERFACE" ]; then
ACTIVE=1
elif [ -x "$WG" ] && "$WG" show "$WG_INTERFACE" >/dev/null 2>&1; then
ACTIVE=1
elif /sbin/ifconfig "$WG_INTERFACE" >/dev/null 2>&1; then
ACTIVE=1
elif pgrep -f "wg-quick up $WG_CONFIG" >/dev/null 2>&1; then
ACTIVE=1
elif pgrep -f "wireguard-go utun" >/dev/null 2>&1; then
ACTIVE=1
fi
echo "active=$ACTIVE"
echo "interfaceName=$WG_INTERFACE"
if [ -n "$RUNTIME_INTERFACE" ]; then
echo "wireguardInterface=$RUNTIME_INTERFACE"
fi
if [ -f "$WG_CONFIG" ]; then
echo "configPath=$WG_CONFIG"
fi
exit 0
fi
if [ "$ACTION" = "cleanup" ]; then
"$WG_QUICK" down "$WG_CONFIG" >/dev/null 2>&1 || true
exit 0
fi
exec "$WG_QUICK" "$ACTION" "$WG_CONFIG"
+3
View File
@@ -0,0 +1,3 @@
node_modules/
dist/
.venv/
+496
View File
@@ -0,0 +1,496 @@
---
name: property-assessor
description: Assess a real property from an address or listing URL and produce a decision-grade summary. Use when a user wants fair value, comps, rental or STR viability, carry-cost review, HOA or insurance risk analysis, or offer guidance for a condo, house, townhouse, or similar residential property. Prefer when the task should discover and reconcile multiple listing sources from a street address first, then give a buy/pass/only-below-X verdict.
---
# Property Assessor
Start from the property address when possible. Treat listing URLs as supporting evidence, not the only source of truth.
## Inputs
Accept any of:
- a street address
- one or more listing URLs
- an address plus the reason for the assessment, such as investment property, vacation home, owner-occupant, long-term rental, STR, or housing for a child in college
The assessment purpose is required for a decision-grade result.
If the user does not say why they want the property assessed, stop and ask before finalizing the analysis.
Do not silently infer or reuse the assessment purpose from earlier turns just because the property address is the same.
Only reuse a prior purpose when the user explicitly says it is the same purpose as before.
If the current request includes only the property/address plus email or PDF instructions, treat the assessment purpose as missing and ask again.
If the current request explicitly says to email or send the PDF to a stated target address, treat that as delivery authorization for that target once the report is ready. Do not ask for a second "send it" confirmation unless the user expressed uncertainty or changed the destination.
If the property has a unit / apartment / suite number, include it.
Do not drop the unit when discovering listing sources. Unit-qualified condo/townhome addresses must be matched as the exact unit, while single-family addresses with no unit should still work normally.
## Core workflow
1. Normalize the address and property type.
2. Discover accessible listing or public-record sources for the same property.
3. Establish a baseline fact set from the best available source.
4. Cross-check the same property on other sites.
5. Pull same-building comps for condos or nearby comps for houses/townhomes.
6. Underwrite carrying cost with taxes, HOA, insurance, and realistic friction.
7. Flag risk drivers before giving a verdict.
8. End with a specific recommendation: `buy`, `pass`, or `only below X`.
Completion rule:
- A preliminary helper payload is not completion.
- If `assess` returns a preliminary payload, continue the remaining analysis yourself in the same run.
- Use the preliminary payload as scaffolding for the missing work: listing facts, comp analysis, valuation range, condition interpretation from photos, and final verdict.
- Do not stop and ask the user whether to continue just because the helper stopped at a preliminary checkpoint.
- Only interrupt the user when there is a genuine blocker: missing assessment purpose, missing recipient email at final render/send time, unavailable listing/public-record sources with no reasonable fallback, or missing facts that cannot be recovered.
## Approval-averse / chat-safe behavior
When operating from chat surfaces such as WhatsApp, Telegram, Signal, or other messaging channels, prefer workflows that do **not** trigger host exec approval prompts.
Use this priority order:
1. `web-automation` for Zillow/HAR listing discovery, rendered listing facts, all-photos flows, and other anti-bot-heavy property pages
2. `web_fetch` when an official CAD/assessor page or simple listing/public page is directly fetchable
3. `web_search` only as a narrow fallback to discover an alternate URL after direct `web-automation` / `web_fetch` attempts miss
4. interactive browser/gallery automation only as a last resort
Rules:
- Prefer accessible listing text, public mirrors, and scrollable all-photos pages over custom gallery traversal.
- Avoid fragile interactive gallery flows if they are likely to require approval or bounce the user into Control UI.
- If a richer photo pass would require approval, do not silently force that path first. Continue with the best approval-free workflow available and clearly lower confidence if needed.
- Only escalate to approval-heavy browser interaction when there is no reasonable alternative and the extra fidelity materially changes the assessment.
- In WhatsApp or similar messaging runs, keep the core assessment on `web-automation` plus `web_fetch`. Treat `web_search` as a fallback discovery aid, not the primary property-analysis path.
- For Zillow/HAR property discovery, photo extraction, and rendered listing review, do **not** start with Brave-backed `web_search` when `web-automation` can open the candidate source directly.
- For CAD/public-record enrichment, prefer official assessor/CAD pages via `web_fetch` first. Use `web-automation` when the official site needs rendered interaction. Do **not** start CAD lookup from generic web-search snippets when the official site is already known or derivable from the address.
- In Texas runs, do **not** use `https://www.texas.gov/propertytaxes/search/` as the CAD lookup path. Use the address-first CAD/helper path or the discovered county CAD pages directly.
- In those messaging runs, do **not** make `scripts/property-assessor assess`, `scripts/property-assessor locate-public-records`, `node zillow-discover.js`, `node har-discover.js`, `node zillow-photos.js`, `node har-photos.js`, `curl`, or `wget` the default core-analysis path.
- From messaging runs, the only subprocess-style step you should attempt by default is the final `scripts/property-assessor render-report` call after the verdict, fair-value range, and report body are complete.
- Do **not** inspect Gmail/email-send skills, mail tooling, or delivery integrations until the assessment is complete and the PDF is either already rendered or ready to render immediately.
- Property-assessor delivery emails should be sent as Luke from Luke's Google Workspace account, while still delivering to the user-specified destination.
- Use `zsh ~/.openclaw/workspace/bin/gog-luke gmail send --to "<target>" --subject "<subject>" --body "<body>" --attach "<pdf-path>"` for the final PDF delivery step.
- Do **not** use generic `gog` or the Stefano helper `node ~/.openclaw/workspace/integrations/google-workspace/gw.js` for property-assessor report delivery.
- If you need to confirm Luke auth before sending, use `zsh ~/.openclaw/workspace/bin/gog-luke auth list --check --plain`.
- A silent helper is a failed helper in messaging runs. If a background helper produces no useful stdout/stderr and no result within a short bound, stop polling it, treat that path as failed, and continue on the chat-native assessment path instead of narrating that it is still chewing.
- If Zillow photo extraction fails, immediately continue with HAR photo fallback or the next available rendered listing/photo source. Do **not** stop the assessment because one Zillow photo subprocess failed.
- After a Zillow/HAR photo miss, continue the comp and CAD/public-record work in the same run. A photo-source miss is a fallback event, not a terminal state.
- Do **not** leave the user parked behind background helper polling. If a helper has not produced a result quickly, give a concise status update and continue the assessment with the next available non-helper path.
- If the user already instructed you to email/send the finished PDF to a specific target, do **not** ask for a second send confirmation after rendering. Render, send, and report the result.
- If the final PDF render fails, return the complete decision-grade report in chat and say the render/send step failed. Do not restart the whole assessment.
- Do **not** run `npm install`, `npm ci`, or other dependency-setup commands during a normal chat assessment flow when the local skill already has `node_modules` present.
- On this machine, treat `property-assessor` dependencies as already installed. Use the helper entrypoints directly unless the wrapper itself explicitly reports missing local dependencies.
- Do **not** use ad hoc shell snippets, heredocs, or inline interpreter eval for public-record or CAD lookup from chat. Avoid forms like `python3 - <<'PY'`, `python -c`, `node -e`, `node --input-type=module -e`, or raw `bash -lc '...'` probes.
- For public-record enrichment in chat, prefer `web_fetch` plus official assessor/CAD pages. If a one-off helper is truly needed, add a file-based helper under the skill tree first and use it from a non-messaging surface instead of inline code.
## Source order
Prefer this order unless the user says otherwise:
1. Zillow
2. Redfin
3. Realtor.com
4. HAR / Homes.com / brokerage mirror pages
5. county or appraisal pages
Use the `web-automation` skill for rendered pages and anti-bot-heavy sites.
Use `web_search` sparingly to discover alternate URLs, then return to `web-automation` for extraction.
## Helper runtime
These helper commands are primarily for local/manual runs, non-chat surfaces, and the final PDF render step. They are **not** the default WhatsApp core-analysis path.
`property-assessor` now includes TypeScript helper commands for:
- address-first preliminary assessment assembly
- public-record jurisdiction lookup
- fixed-template PDF rendering
Before using those helper commands:
```bash
cd ~/.openclaw/workspace/skills/property-assessor
test -x node_modules/.bin/tsx
```
Quick command summary:
```bash
cd ~/.openclaw/workspace/skills/property-assessor
scripts/property-assessor assess --address "<street-address>"
scripts/property-assessor assess --address "<street-address>" --assessment-purpose "<purpose>"
scripts/property-assessor assess --address "<street-address>" --assessment-purpose "<purpose>" --recipient-email "<target@example.com>"
scripts/property-assessor locate-public-records --address "<street-address>"
scripts/property-assessor render-report --input "<report-payload-json>" --output "<output-pdf>"
```
`assess` is the address-first entrypoint for helper-driven runs. It should:
- require the assessment purpose
- treat the assessment purpose as missing unless it is present in the current request or explicitly confirmed as unchanged from earlier context
- resolve official public-record jurisdiction automatically from the address
- keep CAD discovery jurisdiction-specific from the address; do not hardcode one county CAD for every property
- try to discover Zillow and HAR listing URLs from the address when no listing URL is provided
- start Zillow and HAR discovery in parallel, while still preferring Zillow first for the photo-review path
- run the approval-safe Zillow/HAR photo extractor chain automatically
- allow slower exact-unit Zillow pages a longer source-specific discovery/photo window before giving up and falling back
- build a purpose-aware report payload
- complete the analysis without requiring recipient email(s)
- only stop and ask for recipient email(s) when the user is explicitly rendering or sending the PDF
- render the PDF only after recipient email(s) are known
- do **not** render or send a PDF from the helper's preliminary payload while verdict is still `pending` or fair value is not established
- do **not** render or send a decision-grade PDF while `photoReview.status` is anything other than `completed`
- if comps, valuation, or decision-grade condition interpretation are still incomplete, return the preliminary payload and say that the PDF/send step must wait
Agent follow-through rule:
- When the user asked for a full property assessment or asked for the PDF/email result, do not stop at the helper output.
- After `assess` returns a preliminary payload, continue with the remaining manual/model-driven steps needed to reach a decision-grade report.
- Only after the verdict and fair-value range are established should you render/send the PDF.
- A verdict and fair-value range are still not enough by themselves; the subject-unit photo review must also be completed before the PDF/send step.
- If the analysis still cannot be completed, explain the first real blocker, not just that the helper was preliminary.
- If the user sends `update?`, `and?`, or similar mid-run, answer with status and keep the original assessment going. Do not treat that message as a reset or a cue to stop at the last helper checkpoint.
- In WhatsApp or similar messaging runs, do **not** start a background `assess` helper and then wait on repeated zero-output polls. That counts as a failed path; abandon it and continue with native search/fetch/browser work.
- In WhatsApp or similar messaging runs, do **not** start mail-sending or mail-skill discovery before the report content, verdict, and delivery artifact are ready.
- Once the report is ready, treat an explicit prior instruction like "Email pdf version to me at <address>" as sufficient authorization to send.
## Public-record enrichment
Public-record / assessor data should be used when available and linked in the final result.
Default approach:
1. start from the street address
2. resolve the address to county/state/geography
3. identify the related appraisal district / assessor jurisdiction
4. use the official public-record site as a primary-source check against listing data
5. link the official jurisdiction page and any direct property page used in the final result
Approval-safe rule:
- do not perform CAD/public-record discovery with inline shell or interpreter snippets
- in chat/messaging runs, prefer `web_fetch` plus official CAD/assessor pages first
- use the built-in TypeScript helper path on local/manual surfaces or for the final PDF render step
- if rendered interaction is unavoidable, use bounded `web-automation` behavior rather than ad hoc shell text
Use the helper CLI first on local/manual surfaces:
```bash
cd ~/.openclaw/workspace/skills/property-assessor
scripts/property-assessor locate-public-records --address "<street-address>"
```
When you want the helper to assemble the preliminary assessment payload in one step from a non-messaging surface, use:
```bash
cd ~/.openclaw/workspace/skills/property-assessor
scripts/property-assessor assess --address "<street-address>"
```
This command should automatically include the public-record jurisdiction result in the returned assessment payload.
When `--assessment-purpose` is present, it should also:
- frame the analysis around that stated purpose
- try Zillow discovery from the address
- try HAR discovery from the address
- run Zillow photo extraction first when available, then HAR as fallback
- include the extracted `imageUrls` in `photoReview` when successful
This command currently:
- resolves the address through the official Census geocoder
- when Census address matching misses, falls back to an external address geocoder and then resolves official Census geographies from coordinates
- retries the fallback geocoder without the unit suffix when a condo/unit-qualified address is too specific for the fallback provider
- returns county/state/FIPS/GEOID context
- for Texas, resolves the official Texas Comptroller county directory page
- returns the county appraisal district and tax assessor/collector links when available
- when a supported official CAD detail host is found, retrieves subject-property facts from that county CAD and includes them in the assessment payload
Important rules:
- listing-site geo IDs are hints only; do **not** treat them as assessor record keys
- parcel/APN/account identifiers from Zillow/HAR/Redfin are much stronger keys than listing geo IDs
- if Zillow exposes a parcel/APN/account number on the listing, capture it and feed that identifier into CAD lookup before relying on address-only matching
- if a direct public-record property page is available, use its data in the assessment and link it explicitly
- when the helper exposes official CAD owner, legal-description, property-ID/account, value, or exemption data, treat those as primary-source facts in the model's assessment
- if the jurisdiction can be identified but the property detail page is not directly retrievable, still link the official jurisdiction page and say what could not be confirmed
- a host approval prompt triggered by an ad hoc shell snippet is workflow drift; return to `locate-public-records`, `assess`, `web_fetch`, or a file-based helper instead of approving the inline probe by default
### Texas rule
For Texas properties, public-record enrichment is required when feasible.
Process:
1. run `locate-public-records` from the subject address
2. use the returned Texas Comptroller county directory page as the official jurisdiction reference
3. use the returned CAD website for address / account / parcel lookup
- when Zillow exposes the parcel/APN/account number, prefer that over address-only search
4. when accessible, capture:
- account number
- owner name
- land value
- improvement value
- assessed total
- exemptions
- tax office links
In the final assessment, explicitly label official public-record facts as such.
Nueces-specific note:
- when searching Nueces CAD by parcel / Geographic ID, format the identifier with a dash after the first 4 digits and after the first 8 digits, for example `123456789012` -> `1234-5678-9012`
## Minimum data to capture
For the target property, capture when available:
- address
- ask price or last known list price
- property type
- beds / baths
- sqft
- lot size if relevant
- year built
- HOA fee and included services
- taxes
- days on market
- price history
- parking
- waterfront / flood clues
- subdivision / building name when applicable
- same-building or nearby active inventory
- listing photos and visible condition cues
- included appliances and obvious missing appliances
- flooring mix, especially whether carpet is present
- public-record jurisdiction and linked official source
- account / parcel / tax ID if confirmed
- official assessed values and exemptions if confirmed
## Photo and condition review
Always look at the listing photos when they are available. Do not rate a property only from structured text.
### Required photo-access workflow
When the source site exposes listing photos, prefer the **most accessible all-photos view** first. This can be:
- a scrollable all-photos page
- a photo grid
- an expanded photo list
- or, if necessary, a modal gallery/lightbox
Use `web-automation` for this. Preferred process:
1. Open the listing page.
2. Click the photo entry point such as `See all photos`, `See all 29 photos`, `Show all photos`, or the main hero image.
3. If that opens a scrollable all-photos view, grid, or photo page that clearly exposes the listing images, use that directly for photo review.
4. Only use next-arrow / slideshow traversal when the site does not provide an accessible all-photos view.
5. If you must use a modal/lightbox, verify that you are seeing distinct images, not just a gallery preview tile or repeated screenshot of the first image.
6. Review enough images to cover the key rooms and exterior, and for smaller listings aim to review all photos when practical.
7. If photo access fails or is incomplete, say so explicitly and do not claim that you reviewed all photos.
Minimum honesty rule: never say you "looked at all photos" unless the site actually exposed the full set and you successfully reviewed them.
A gallery landing page, collage preview, repeated first image, or a single screenshot of the listing page does **not** count as full photo review.
### What to inspect in the photos
At minimum, note:
- overall finish level: dated, average, lightly updated, fully updated
- kitchen condition: cabinets, counters, backsplash, appliance quality
- bathroom condition: vanity, tile, surrounds, fixtures
- flooring: tile, vinyl, laminate, hardwood, carpet
- whether carpet appears in bedrooms, stairs, or living areas
- obvious make-ready issues: paint, damaged trim, old fixtures, mismatched finishes, worn surfaces
- visible missing items: refrigerator, washer/dryer, range hood, dishwasher, etc.
- any signs of deferred maintenance or water intrusion visible in photos
- exterior/common-area condition when visible
- balconies, decks, sliders, windows, and waterfront-facing elements for condos/townhomes near water
### If photo review is incomplete
If photos are weak, incomplete, blocked, or the gallery automation fails:
- say so explicitly
- lower confidence
- avoid strong condition claims
- do not infer turnkey condition from marketing text alone
### Mandatory photo-review rule
If an accessible all-photos view, photo grid, photo page, or fallback source exists, photo review is **required** before making condition claims.
Do not silently skip photos just because pricing, comps, or carrying-cost analysis can proceed without them.
Before outputting `Photo review: not completed`, you must attempt a reasonable photo-access chain when sources are available.
Preferred order:
1. primary listing source all-photos page (for example Zillow `See all photos` / `See all X photos`)
2. HAR photo page
3. Realtor.com photo page
4. brokerage mirror or other accessible listing mirror
Use the first source that exposes the listing photos reliably. A scrollable photo page, photo grid, or expanded all-photos view counts.
Do not stop at the first failure if another accessible source is available.
### Zillow-specific rule
For Zillow listings, do **not** treat the listing shell text or hero gallery preview as a photo review attempt.
You must use **only `web-automation`** to:
1. open the Zillow listing page
2. click `See all photos` / `See all X photos`
3. access the resulting all-photos page or scrollable photo view
4. review the exposed photo set from that page
If Zillow exposes a page with a scroller that shows the listing photos, that page counts as the Zillow photo source and should be used directly.
If that scroller page exposes direct image links such as `https://photos.zillowstatic.com/...`, treat those URLs as successful photo access and use that image set for review.
This is preferred over fragile modal next/previous navigation.
If the rendered Zillow listing shell itself already exposes the full direct Zillow image set and the extracted image count matches the announced photo count, that also counts as successful photo access even if the `See all photos` click path is flaky.
For this normal Zillow all-photos workflow, stay inside **`web-automation` only**.
Use the dedicated file-based extractor first:
```bash
cd ~/.openclaw/workspace/skills/web-automation/scripts
node zillow-photos.js "<zillow-listing-url>"
```
Do **not** escalate to coding-agent, ad hoc Python helpers, or extra dependency-heavy tooling just to open `See all photos`, inspect the scroller page, or extract Zillow image URLs.
Only escalate beyond `web-automation` if `web-automation` itself truly cannot access the all-photos/scroller page or the exposed image set.
Do not rely on generic page text, photo counts, or non-photo shells as a Zillow attempt.
Only fall back to HAR/Realtor/broker mirrors if the Zillow all-photos path was actually attempted with `web-automation` and did not expose the photos reliably.
A source only counts as an attempted photo source if you actually did one of these:
- opened the all-photos page / photo grid / photo page successfully, or
- explicitly tried to open it and observed a concrete failure
The following do **not** count as a photo-source attempt by themselves:
- seeing a `See all photos` button
- seeing a photo count
- reading listing text that mentions photos
- capturing only the listing shell, hero image, or collage preview
### HAR fallback rule
If Zillow photo extraction does not expose usable direct image URLs, try HAR next for the same property.
For HAR listings, use **only `web-automation`** to:
1. open the HAR listing page
2. click `Show all photos` / `View all photos`
3. access the resulting all-photos page or photo view
4. extract the direct image URLs from that page
Use the dedicated HAR extractor first:
```bash
cd ~/.openclaw/workspace/skills/web-automation/scripts
node har-photos.js "<har-listing-url>"
```
If HAR exposes the direct photo URLs from the all-photos page, treat that as successful photo access and use that image set for review.
Do not stop after a failed Zillow attempt if HAR is available and exposes the listing photos more reliably.
When a dedicated extractor returns `imageUrls`, inspect the images in that returned set before making condition claims.
For smaller listings, review the full extracted set when practical; for a 20-30 photo listing, that usually means all photos.
### Approval-safe command shape
When running `web-automation` from chat-driven property assessment, prefer file-based commands under `~/.openclaw/workspace/skills/web-automation/scripts`.
Good:
- `node check-install.js`
- `node zillow-photos.js "<url>"`
- `node har-photos.js "<url>"`
Avoid approval-sensitive inline interpreter eval where possible:
- `node -e "..."`
- `node --input-type=module -e "..."`
The final assessment must explicitly include these lines in the output:
- `Photo source attempts: <action-based summary>`
- `Photo review: completed via <source>`
or `Photo review: not completed`
If completed, briefly summarize the condition read from the photos.
If not completed, mark condition confidence as limited and say why.
## PDF report requirement
The deliverable is not just chat text. A fixed-template PDF report must be generated for completed assessments.
Use the property-assessor helper CLI:
```bash
cd ~/.openclaw/workspace/skills/property-assessor
npm install
scripts/property-assessor assess --address "<street-address>" --recipient-email "<target@example.com>"
scripts/property-assessor render-report --input "<report-payload-json>" --output "<output-pdf>"
```
The renderer uses a fixed template and must keep the same look across runs.
Template rules are documented in `references/report-template.md`.
The PDF report should include:
1. report header
2. verdict panel
3. subject-property summary table
4. Snapshot
5. What I like
6. What I do not like
7. Comp view
8. Underwriting / carry view
9. Risks and diligence items
10. Photo review
11. Public records
12. Source links
### Recipient-email gate
Before rendering or sending the PDF, the skill must know the target recipient email address(es).
This gate applies to the PDF step, not the analysis step.
Do not interrupt a normal assessment run just because recipient email is missing if the user has not yet asked to render or send the PDF.
If the prompt does **not** include target email(s):
- stop
- ask the user for the target email address(es)
- do **not** render or send the final PDF yet
If target email(s) are present:
- include them in the report payload
- render the PDF with the fixed template
- if a delivery workflow is available, use those same target email(s) for sending
The renderer enforces this gate and will fail if the payload has no recipient email list.
## Normalization / make-ready adjustment
Estimate a rough make-ready budget when condition is not turnkey. The goal is not contractor precision; the goal is apples-to-apples comparison.
Use simple buckets and state them as rough ranges:
- light make-ready: paint, fixtures, minor hardware, patching
- medium make-ready: flooring replacement in some rooms, appliance replacement, bathroom refresh
- heavy make-ready: major kitchen/bath work, widespread flooring, obvious deferred maintenance
Call out carpet separately. If carpet is present, estimate replacement or removal cost as part of the make-ready note.
## Underwriting rules
Always show a simple carrying-cost view with at least:
- principal and interest if available from the listing
- taxes per month
- HOA per month if applicable
- insurance estimate or note uncertainty
- realistic effective carry range after maintenance, vacancy, and property-specific risk
Treat these as strong caution flags:
- high HOA relative to price or expected rent
- older waterfront or coastal exposure
- unknown reserve / assessment history for condos
- many active units in the same building or micro-area
- stale days on market with weak price action
- no clear rent support
## Output format
Keep the answer concise but decision-grade:
1. Snapshot
2. What I like
3. What I do not like
4. Comp view
5. Underwriting / carry view
6. Risks and diligence items
7. Verdict with fair value range and offer guidance
Also include:
- public-record / CAD evidence and links when available
- the path to the rendered PDF after generation
If the user did not provide recipient email(s), ask for them instead of finalizing the PDF workflow.
## Reuse notes
When condos are involved, same-building comps and HOA economics usually matter more than neighborhood averages.
For detailed heuristics and the reusable memo template, read:
- `references/underwriting-rules.md`
- `references/report-template.md`

Some files were not shown because too many files have changed in this diff Show More