82 Commits

Author SHA1 Message Date
stefano 0bea1c590d fix: review feedback — signal handling, cancel race, stderr consistency
Address issues found by code review:

1. Bug: timeout/signal-killed child reported as 'completed' with exit
   code 0 because close handler ignored the signal parameter. Now
   treats any signal termination as timed_out.

2. Bug: cancelled job gets overwritten by watcher on child exit. The
   watcher now re-reads the job file before writing and skips if the
   status has been changed to 'cancelled'.

3. Inconsistency: watcher path skipped stderr noise filtering. Added
   filterStderrNoise to the watcher (duplicated from execute.ts to
   keep the watcher self-contained).

4. getJobResult now guards against missing result field instead of
   using non-null assertion.
2026-05-20 14:17:28 -05:00
stefano 33c898ff9a fix: use companion watcher process for async job completion
The async startJob previously relied on Node.js event listeners in the
CLI process to capture child output and finalize the job file. But the
CLI process exits immediately after returning the job ID, killing the
event loop before the close handler fires — leaving jobs stuck at
'running' forever.

Fix: startJob now spawns a companion watcher process (job-watcher.ts)
that is itself detached and outlives the CLI. The watcher:
- Spawns the actual client (codex/claude/opencode)
- Captures stdout/stderr
- Writes the final job record to disk on child exit
- Has its own 10-minute timeout safety net

The CLI returns the job ID immediately. The watcher independently
finalizes the job. The CLI no longer needs to stay alive.

Also updates tests to mock the watcher spawn via injectable
spawnWatcher option.
2026-05-20 14:08:44 -05:00
stefano 017eb1b410 fix: pipe and close stdin for codex to prevent hang on stdin read
When codex exec receives a prompt as a positional argument, it still
tries to read additional input from stdin (prints 'Reading additional
input from stdin...'). With stdio stdin set to 'ignore' or default,
codex blocks indefinitely waiting for stdin that never comes.

Fix: use stdio ['pipe', 'pipe', 'pipe'] and immediately close stdin
via child.stdin.end() in both execute.ts (sync) and jobs.ts (async).
This signals EOF to codex so it proceeds with the positional prompt.
2026-05-20 13:47:32 -05:00
stefano afac143cb3 fix: filter codex ReasoningSummary stderr noise on exit code 0
Codex writes informational ERROR messages about ReasoningSummaryDelta
to stderr even on successful execution (exit code 0). The OpenClaw
agent misinterprets this non-empty stderr as a failure.

- Add filterStderrNoise() to strip known codex noise patterns from
  stderr when exit code is 0
- Preserve raw stderr in DebugInfo.rawStderr when --debug is active
- Add 5 new tests covering noise filtering, preservation on failure,
  debug raw output, and non-codex client passthrough
2026-05-20 13:37:21 -05:00
stefano edb6611b74 merge M4 into implement/2026-05-19-ai-cli-dispatch-fixes 2026-05-19 22:49:05 -05:00
stefano 7b886a7b33 feat(M4): Documentation & Final Integration 2026-05-19 22:49:05 -05:00
stefano 48bef5cc7c merge S-402 into M4 2026-05-19 22:45:38 -05:00
stefano e6f2908624 merge S-401 into M4 2026-05-19 22:45:38 -05:00
stefano 601f7cce89 feat(S-402): Update docs/ai-cli-dispatch.md and docs/architecture.md 2026-05-19 22:45:38 -05:00
stefano 6655e2e1e8 feat(S-401): Update SKILL.md for async-first usage 2026-05-19 22:42:16 -05:00
stefano bd88df7dd2 merge M3 into implement/2026-05-19-ai-cli-dispatch-fixes 2026-05-19 22:22:54 -05:00
stefano 591829369c feat(M3): Async CLI Integration 2026-05-19 22:22:54 -05:00
stefano a2c2b8bf6d merge S-303 into M3 2026-05-19 22:04:19 -05:00
stefano 51f978db4c feat(S-303): Update --help output and add CLI integration smoke tests 2026-05-19 22:04:19 -05:00
stefano d061244121 merge S-302 into M3 2026-05-19 22:00:11 -05:00
stefano 4fe99b8c57 feat(S-302): Test-drive and implement job lifecycle subcommands 2026-05-19 22:00:11 -05:00
stefano 816374cef8 merge S-301 into M3 2026-05-19 21:42:58 -05:00
stefano 62840b908e feat(S-301): Test-drive and implement async default for run and dispatch 2026-05-19 21:42:58 -05:00
stefano e11c36b7d8 merge M2 into implement/2026-05-19-ai-cli-dispatch-fixes 2026-05-19 20:29:35 -05:00
stefano e7b01612c8 feat(M2): Background Job Manager 2026-05-19 20:29:35 -05:00
stefano 9c7d9cbaee merge S-202 into M2 2026-05-19 20:17:15 -05:00
stefano 3b9ed0cc38 feat(S-202): Test-drive and implement src/jobs.ts (write) 2026-05-19 20:17:15 -05:00
stefano aa860a6afd merge S-201 into M2 2026-05-19 19:58:48 -05:00
stefano abf7726071 feat(S-201): Define job types and storage interfaces 2026-05-19 19:58:48 -05:00
stefano 21c13562a7 merge M1 into implement/2026-05-19-ai-cli-dispatch-fixes 2026-05-19 19:54:27 -05:00
stefano bcddb42608 feat(M1): Codex Reliability Fix 2026-05-19 19:54:27 -05:00
stefano 5b78889b09 merge S-104 into M1 2026-05-19 19:51:10 -05:00
stefano 1983dd82e7 feat(S-104): Add stderr-length and exit-code correlation diagnostics 2026-05-19 19:51:10 -05:00
stefano 106c7d6425 merge S-103 into M1 2026-05-19 19:48:40 -05:00
stefano 94389df6f1 feat(S-103): Test-drive and implement --debug diagnostic mode 2026-05-19 19:48:40 -05:00
stefano 32964bf994 merge S-102 into M1 2026-05-19 19:39:46 -05:00
stefano dc3fe8d6eb feat(S-102): Test-drive and implement --timeout flag, config layering, and default in 2026-05-19 19:39:46 -05:00
stefano 5375c83c77 merge S-101 into M1 2026-05-19 19:20:53 -05:00
stefano 476dd317b3 feat(S-101): Extend types.ts with ExecResult metadata, timeout config shape, and debu 2026-05-19 19:20:53 -05:00
stefano e523b34d1b fix: codex uses --yolo not --full-auto 2026-05-18 19:15:59 -05:00
stefano fd1d2c3e92 fix: invoke all CLI clients in full-access/yolo mode
- codex: --full-auto
- claude: --dangerously-skip-permissions
- opencode: --dangerously-skip-permissions
2026-05-18 19:15:04 -05:00
stefano d3aa92be0d fix: use 'opencode run' instead of bare prompt for OpenCode client 2026-05-18 19:06:34 -05:00
stefano 0e273b59ec chore: add ai_plan/ to .gitignore 2026-05-18 18:51:32 -05:00
stefano 2e884e49c8 merge M6 into implement/2026-05-18-stef-openclaw-skills 2026-05-18 18:44:07 -05:00
stefano 775a665eaa feat(M6): Documentation 2026-05-18 18:44:07 -05:00
stefano 2103c424f4 merge S-604 into M6 2026-05-18 18:43:58 -05:00
stefano 32f8a23700 merge S-603 into M6 2026-05-18 18:43:58 -05:00
stefano 480958f12e feat(S-603): Create docs/architecture.md 2026-05-18 18:43:58 -05:00
stefano c188f09684 feat(S-604): Update README.md and docs/README.md 2026-05-18 18:43:08 -05:00
stefano 7818e78244 merge S-602 into M6 2026-05-18 18:41:58 -05:00
stefano c35ffe8af5 merge S-601 into M6 2026-05-18 18:41:58 -05:00
stefano a6f855c9d9 feat(S-602): Create docs/installation.md 2026-05-18 18:41:58 -05:00
stefano 52675f6dc1 feat(S-601): Create docs/ai-cli-dispatch.md 2026-05-18 18:41:58 -05:00
stefano d87038204b merge M5 into implement/2026-05-18-stef-openclaw-skills 2026-05-18 18:39:33 -05:00
stefano 4f59258b20 feat(M5): CLI Integration 2026-05-18 18:39:33 -05:00
stefano 0879ffe39f merge M4 into implement/2026-05-18-stef-openclaw-skills 2026-05-18 18:14:48 -05:00
stefano fe7a015ca4 feat(M4): Natural Language Dispatch 2026-05-18 18:14:48 -05:00
stefano 0c9248d5ca merge S-401 into M4 2026-05-18 18:14:13 -05:00
stefano 7fa959d115 feat(S-401): Test-drive and implement src/dispatch.ts 2026-05-18 18:14:13 -05:00
stefano 50443373bd merge M3 into implement/2026-05-18-stef-openclaw-skills 2026-05-18 18:11:45 -05:00
stefano a2cfa7027e feat(M3): Direct Execution 2026-05-18 18:11:45 -05:00
stefano fe94629797 merge S-301 into M3 2026-05-18 18:01:51 -05:00
stefano a99041f910 feat(S-301): Test-drive and implement src/execute.ts 2026-05-18 18:01:51 -05:00
stefano 360e27d952 merge M2 into implement/2026-05-18-stef-openclaw-skills 2026-05-18 17:53:47 -05:00
stefano 82fcd3363c feat(M2): Client Detection & Configuration 2026-05-18 17:53:47 -05:00
stefano 185083ace8 merge S-203 into M2 2026-05-18 17:51:44 -05:00
stefano 167cdb6ffe merge S-202 into M2 2026-05-18 17:51:44 -05:00
stefano f3458734d4 feat(S-203): Test-drive and implement src/config.ts 2026-05-18 17:51:44 -05:00
stefano 2642c280a2 feat(S-202): Test-drive and implement src/detect.ts 2026-05-18 17:51:07 -05:00
stefano 8340933f8a merge M1 into implement/2026-05-18-stef-openclaw-skills 2026-05-18 17:45:27 -05:00
stefano 4629fe17de feat(M1): Project Scaffold 2026-05-18 17:45:27 -05:00
stefano 949bd05420 merge S-103 into M1 2026-05-18 17:40:10 -05:00
stefano 162517c0e0 feat(S-103): Create scripts/ai-cli-dispatch launcher 2026-05-18 17:40:10 -05:00
stefano 4a6cacb21d merge S-201 into M1 2026-05-18 17:38:35 -05:00
stefano 47f555a367 feat(S-201): Create src/types.ts with shared type definitions 2026-05-18 17:38:35 -05:00
stefano 445d9bfdee merge S-102 into M1 2026-05-18 17:36:31 -05:00
stefano 28e6bbba74 merge S-101 into M1 2026-05-18 17:36:31 -05:00
stefano 50928313a1 feat(S-102): Create package.json, tsconfig.json, .gitignore 2026-05-18 17:36:31 -05:00
stefano fb01334273 feat(S-101): Create SKILL.md with YAML frontmatter 2026-05-18 17:36:21 -05:00
stefano b3875858c7 fix(amazon-shopping): enforce rating filters in chat output 2026-04-15 21:05:27 -05:00
stefano fda0602ac9 fix(amazon-shopping): verify prime and delivery filters 2026-04-15 20:28:16 -05:00
stefano a81a055ec6 docs(amazon-shopping): add install and update instructions 2026-04-15 19:19:09 -05:00
stefano 4204d28077 fix(amazon-shopping): harden agent invocation 2026-04-15 19:09:52 -05:00
stefano c1286e9c42 docs(amazon-shopping): document amazon product search skill 2026-04-15 18:51:42 -05:00
stefano 1e0e265f1e feat(amazon-shopping): scrape and filter amazon product results 2026-04-15 18:48:51 -05:00
stefano ef326896f4 feat(amazon-shopping): parse filters and extract search candidates 2026-04-15 18:31:44 -05:00
stefano 8ad532545d feat(amazon-shopping): scaffold amazon product search skill 2026-04-15 18:24:13 -05:00
64 changed files with 9704 additions and 0 deletions
+1
View File
@@ -1,2 +1,3 @@
.worktrees/
node_modules/
ai_plan/
+9
View File
@@ -11,11 +11,14 @@ This repository contains practical OpenClaw skills and companion integrations. I
- Per-skill runtime instructions: `skills/<skill-name>/SKILL.md`
- Integration implementation files: `integrations/<integration-name>/`
- Integration docs: `docs/*.md`
- Tool implementation files: `tools/<tool-name>/`
- Tool docs: `docs/*.md`
## Skills
| Skill | What it does | Path |
|---|---|---|
| `amazon-shopping` | Search Amazon.com product results with bounded web-automation, price/unit-price/width/Prime/delivery filters, specs, ratings, review metadata, and chat-safe result blocks with direct links. | `skills/amazon-shopping` |
| `flight-finder` | Collect structured flight-search inputs, run bounded multi-source travel searches, rank results in USD, and gate fixed-template PDF/email delivery through Lukes sender path. | `skills/flight-finder` |
| `gitea-api` | Interact with Gitea via REST API (repos, issues, PRs, releases, branches, user info). | `skills/gitea-api` |
| `nordvpn-client` | Install, log in to, connect, disconnect, and verify NordVPN sessions across Linux CLI and macOS NordLynx/WireGuard backends. | `skills/nordvpn-client` |
@@ -33,6 +36,12 @@ This repository contains practical OpenClaw skills and companion integrations. I
| `google-maps` | Traffic-aware ETA and leave-by calculations using Google Maps APIs. | `integrations/google-maps` |
| `google-workspace` | Gmail and Google Calendar helper CLI for profile, mail, attachment-capable send, calendar search, and event creation. | `integrations/google-workspace` |
## Tools
| Tool | What it does | Path |
|---|---|---|
| `ai-cli-dispatch` | Dispatch AI CLI coding tasks to available clients (Codex, Claude Code, OpenCode) with automatic discovery, version checking, and execution. | `tools/ai-cli-dispatch` |
## Operator docs
| Doc | What it covers |
+7
View File
@@ -4,6 +4,7 @@ This folder contains detailed docs for each skill in this repository.
## Skills
- [`amazon-shopping`](amazon-shopping.md) — Amazon.com product search with local web-automation, product filters, pricing, delivery, specs, and review metadata
- [`flight-finder`](flight-finder.md) — Typed flight-search intake, bounded source orchestration, PDF report rendering, and Luke-sender email delivery
- [`gitea-api`](gitea-api.md) — REST-based Gitea automation (no `tea` CLI required)
- [`nordvpn-client`](nordvpn-client.md) — Cross-platform NordVPN install, login, connect, disconnect, and verification with Linux CLI and macOS NordLynx/WireGuard support
@@ -19,6 +20,12 @@ This folder contains detailed docs for each skill in this repository.
- [`google-maps`](google-maps.md) — Traffic-aware ETA and leave-by calculations via Google Maps APIs
- [`google-workspace`](google-workspace.md) — Gmail and Google Calendar helper CLI with attachment-capable send support
## Tools
- [`ai-cli-dispatch`](ai-cli-dispatch.md) — Dispatch AI CLI coding tasks to available clients with automatic discovery, version checking, and execution
- [`installation`](installation.md) — Prerequisites, install steps, PATH configuration, and optional config file setup for `ai-cli-dispatch`
- [`architecture`](architecture.md) — Design decisions, module breakdown, data flow, coexistence with ACP, and extension points for `ai-cli-dispatch`
## Operator Docs
- [`openclaw-acp-orchestration`](openclaw-acp-orchestration.md) — OpenClaw ACP orchestration for Codex and Claude Code on the gateway host
+460
View File
@@ -0,0 +1,460 @@
# ai-cli-dispatch
Dispatch AI CLI coding tasks to available clients (Codex, Claude Code, OpenCode) with automatic discovery, version checking, and execution.
## Scope
- discover installed AI CLI clients on the host
- report client versions and availability
- dispatch a prompt to a specific client by name
- auto-resolve the best client from prompt keywords
- forward arguments natively to each client
- run tasks asynchronously as background jobs with lifecycle management
- run tasks synchronously when blocking until completion is desired
The tool supports both async (default) and sync execution modes. Async jobs run as detached background processes and are tracked on disk. For ACP-based harnesses, see `docs/openclaw-acp-orchestration.md`.
## Setup
From the repo or installed skill directory:
```bash
cd tools/ai-cli-dispatch
npm install
```
The dispatcher itself requires only Node.js 20+ and `npm`. The actual AI CLI clients (`codex`, `claude`, `opencode`) are discovered from the host `PATH`; they are not bundled.
## Commands
```bash
ai-cli-dispatch list [--json|--text]
ai-cli-dispatch run --client <client> --prompt <prompt> [--sync] [--timeout <ms>] [--debug] [--json|--text]
ai-cli-dispatch dispatch <prompt> [--client <client>] [--sync] [--timeout <ms>] [--debug] [--json|--text]
ai-cli-dispatch start --client <client> --prompt <prompt> [--timeout <ms>] [--debug] [--json|--text]
ai-cli-dispatch status <job-id> [--json|--text]
ai-cli-dispatch results <job-id> [--json|--text]
ai-cli-dispatch cancel <job-id> [--json|--text]
ai-cli-dispatch list-jobs [--status running|completed|failed] [--json|--text]
ai-cli-dispatch cleanup-jobs [--max-age <number>[h|m|s|d]] [--json|--text]
ai-cli-dispatch --help
```
### `list`
Discover and report all supported clients.
```bash
ai-cli-dispatch list --json
```
Example JSON output:
```json
[
{
"name": "codex",
"path": "/usr/local/bin/codex",
"version": "1.2.3",
"found": true
},
{
"name": "claude",
"found": false
},
{
"name": "opencode",
"path": "/opt/homebrew/bin/opencode",
"version": "0.5.1",
"found": true
}
]
```
Use `--text` for human-readable checkmarks:
```bash
ai-cli-dispatch list --text
```
### `run`
Execute a prompt directly through a named client. By default, this starts a background job and returns immediately.
```bash
# Async (default) — returns a job ID
ai-cli-dispatch run --client codex --prompt "refactor this function"
# Sync — blocks until the client finishes
ai-cli-dispatch run --client claude --prompt "add tests for auth middleware" --sync
# With custom timeout and debug diagnostics
ai-cli-dispatch run --client opencode --prompt "migrate to ESM" --timeout 600000 --debug
```
The prompt is forwarded with each clients native argument shape:
| Client | Arguments passed |
|---|---|
| `codex` | `exec --yolo "<prompt>"` |
| `claude` | `-p "<prompt>" --dangerously-skip-permissions` |
| `opencode` | `run --dangerously-skip-permissions "<prompt>"` |
### `dispatch`
Auto-resolve the client from prompt keywords, then execute. Defaults to async; use `--sync` to block.
```bash
ai-cli-dispatch dispatch "use claude to write tests"
ai-cli-dispatch dispatch "codex refactor auth module"
ai-cli-dispatch dispatch "opencode migrate to ESM"
```
Keyword matching is case-insensitive and ordered:
1. `--client` flag (highest precedence)
2. `"open code"` (spaced variant) → `opencode`
3. `"claude"``claude`
4. `"codex"``codex`
5. `"opencode"``opencode`
6. `defaultClient` from config (lowest precedence)
Override auto-resolution explicitly:
```bash
ai-cli-dispatch dispatch "fix the bug" --client claude
```
### `start`
Explicitly start a background job (same as `run` without `--sync`). Useful when you want the async behavior unambiguously.
```bash
ai-cli-dispatch start --client codex --prompt "refactor this function"
```
### `status`
Check the status of a background job.
```bash
ai-cli-dispatch status <job-id>
```
JSON output:
```json
{
"id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"client": "codex",
"prompt": "refactor this function",
"status": "running",
"startedAt": "2026-05-19T12:34:56.789Z",
"pid": 12345
}
```
Statuses: `running`, `completed`, `failed`, `timed_out`, `cancelled`.
### `results`
Retrieve the execution result of a completed job.
```bash
ai-cli-dispatch results <job-id>
```
JSON output:
```json
{
"stdout": "...",
"stderr": "...",
"exitCode": 0,
"client": "codex",
"durationMs": 45231
}
```
Requires status `completed`. For `failed` or `timed_out` jobs, use `status` to see the captured error.
### `cancel`
Cancel a running job.
```bash
ai-cli-dispatch cancel <job-id>
```
### `list-jobs`
List all tracked jobs, newest first.
```bash
ai-cli-dispatch list-jobs --json
ai-cli-dispatch list-jobs --status running --json
```
### `cleanup-jobs`
Remove job files older than a threshold. Default unit is hours.
```bash
ai-cli-dispatch cleanup-jobs --max-age 24h
ai-cli-dispatch cleanup-jobs --max-age 30m
```
## Async vs Sync Mode
By default, `run` and `dispatch` are **async**: they start a detached background process, persist a job record to disk, and return a job ID immediately. This is ideal for:
- Fire-and-forget tasks that may run for minutes
- Long-running codegen or migration tasks
- Scenarios where the caller should not block
Use `--sync` when you need:
- The complete output before the next step
- Synchronous composition in shell pipelines or scripts
- Immediate error propagation to the calling process
| Aspect | Async (default) | Sync (`--sync`) |
|---|---|---|
| Return value | Job ID + status | Full stdout/stderr + exit code |
| Process model | Detached child, parent exits immediately | Attached child, parent waits |
| Persistence | Job file written to disk | No job file |
| Timeout | Enforced via `child.kill()` after `--timeout` | Enforced via `child.kill()` after `--timeout` |
## Client Discovery
Discovery searches `PATH` in this order for each client name:
1. `codex` — OpenAI Codex CLI
2. `claude` — Anthropic Claude Code
3. `opencode` — OpenCode CLI
The search uses `which` (or `where` on Windows) first, then falls back to a manual `PATH` directory scan. If a binary is found, `--version` is invoked to extract a semver string.
## Configuration
Optional config file:
```text
~/.openclaw/ai-cli-dispatch.json
```
Example:
```json
{
"paths": {
"codex": "/usr/local/bin/codex",
"claude": "/opt/homebrew/bin/claude"
},
"defaultClient": "claude",
"timeout": 300000
}
```
Resolution priority for paths and default client (highest to lowest):
1. CLI flag (`--client`, `--codex-path`, etc.)
2. Environment variable (`AI_CLI_CODEX_PATH`, `AI_CLI_DEFAULT_CLIENT`, etc.)
3. Config file (`paths`, `defaultClient`)
4. `which` / `where` discovery
Supported env vars:
| Variable | Purpose |
|---|---|
| `AI_CLI_CODEX_PATH` | Override `codex` binary path |
| `AI_CLI_CLAUDE_PATH` | Override `claude` binary path |
| `AI_CLI_OPENCODE_PATH` | Override `opencode` binary path |
| `AI_CLI_DEFAULT_CLIENT` | Override default client (`codex`, `claude`, or `opencode`) |
## Output Model
Default output is JSON. Use `--text` to stream raw `stdout`/`stderr` directly.
### Sync JSON success shape (`run --sync`, `dispatch --sync`)
```json
{
"stdout": "...",
"stderr": "...",
"exitCode": 0,
"client": "codex",
"durationMs": 45231
}
```
### Async JSON success shape (`run`, `dispatch`, `start`)
```json
{
"jobId": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"client": "codex",
"status": "running"
}
```
### Job status shape (`status`)
```json
{
"id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"client": "codex",
"prompt": "refactor this function",
"status": "running",
"startedAt": "2026-05-19T12:34:56.789Z",
"pid": 12345
}
```
### Job result shape (`results`)
```json
{
"stdout": "...",
"stderr": "...",
"exitCode": 0,
"client": "codex",
"durationMs": 45231
}
```
### JSON error shape
```json
{
"error": "..."
}
```
Exit codes:
| Code | Meaning |
|---|---|
| `0` | Success |
| `1` | Missing/unknown command, missing argument, unknown client, resolution failure, execution error, or job lifecycle error |
## Error Handling Guidance
### `Client "<name>" not found or not installed`
Meaning: the requested client binary is not on `PATH` and not overridden by config.
Actions:
1. Confirm the client is installed (`codex --version`, `claude --version`, etc.)
2. Check that its directory is on `PATH`
3. Or override the path in `~/.openclaw/ai-cli-dispatch.json`
### `Prompt cannot be empty`
Meaning: the prompt string was empty or whitespace-only.
Action: supply a non-empty `--prompt` or positional prompt argument.
### `Execution timed out after 600000ms`
Meaning: the client subprocess did not finish within the timeout.
Action: the client may be waiting for interactive input or the task is too large. Break the prompt into smaller pieces, increase `--timeout`, or run the client directly to diagnose. Async jobs that time out are recorded with status `timed_out`.
### `Could not resolve client from prompt`
Meaning: `dispatch` found no matching keyword and no `defaultClient` is configured.
Action: include a client name in the prompt (e.g., `"use claude to ..."`) or set `defaultClient` in config.
### `Job "<job-id>" not found`
Meaning: the requested job ID does not exist in the job store.
Action: verify the job ID. Job files are stored under `~/.openclaw/ai-cli-dispatch/jobs/`. If the directory was cleaned or the host restarted, old jobs may have been removed.
### `Job "<job-id>" result is not available (status: <status>)`
Meaning: `results` was called on a job that has not finished (`running`) or finished unsuccessfully (`failed`, `timed_out`, `cancelled`).
Action: poll `status` until the job reaches `completed`, or inspect `status` output for the error field.
## Job Lifecycle Workflows
### Fire-and-forget
```bash
JOB=$(ai-cli-dispatch run --client codex --prompt "refactor auth" --json | jq -r '.jobId')
# caller continues immediately
```
### Poll until completion
```bash
JOB=$(ai-cli-dispatch start --client claude --prompt "write tests" --json | jq -r '.jobId')
while [ "$(ai-cli-dispatch status "$JOB" --json | jq -r '.status')" = "running" ]; do
sleep 5
done
ai-cli-dispatch results "$JOB" --json
```
### Sync one-shot
```bash
ai-cli-dispatch run --client opencode --prompt "fix lint" --sync --text
```
### Batch cleanup
```bash
ai-cli-dispatch cleanup-jobs --max-age 24h
```
## Common Flows
### Check what is installed
```bash
ai-cli-dispatch list --json
```
### Run a quick task through a specific client (async)
```bash
ai-cli-dispatch run --client codex --prompt "fix lint errors in src/app.ts"
```
### Run a quick task synchronously
```bash
ai-cli-dispatch run --client codex --prompt "fix lint errors in src/app.ts" --sync
```
### Let the tool pick the client from the prompt
```bash
ai-cli-dispatch dispatch "claude: add unit tests for utils.ts"
```
### Force a client when the prompt is ambiguous
```bash
ai-cli-dispatch dispatch "review this PR" --client claude
```
## Coexistence with ACP
`ai-cli-dispatch` is a direct subprocess dispatcher. It is not an ACP agent and does not participate in ACP orchestration.
- Use `ai-cli-dispatch` when you need a quick, local, one-shot CLI execution or a background job.
- Use ACP (`docs/openclaw-acp-orchestration.md`) when you need session-bound coding harnesses with thread context, multi-turn review, or orchestrator-managed verification gates.
## Implementation Notes
- The dispatcher is TypeScript/Node.js with a single external dependency (`minimist`).
- Client arguments are hardcoded per tool to match each clients stable CLI contract.
- The default timeout is 10 minutes (`600_000` ms); override with `--timeout` or config.
- On Windows, discovery uses `where` instead of `which` and `.exe` extensions are assumed.
- Async jobs run as detached processes with `stdio: ["ignore", "pipe", "pipe"]` so the dispatcher can exit without waiting.
- Job files are written atomically to `~/.openclaw/ai-cli-dispatch/jobs/<jobId>.json`.
+146
View File
@@ -0,0 +1,146 @@
# Amazon Shopping Skill
`amazon-shopping` searches Amazon.com product results with bounded, read-only web automation and deterministic local filtering.
## Example Invocation
```text
use amazon-shopping to search for 100w led bulbs that cost less than $4 each and have over 200 reviews with a review score of more than 4.5 stars
```
## Helper Commands
Run from the installed skill:
```bash
cd ~/.openclaw/workspace/skills/amazon-shopping
scripts/search-products 'USB-C charger under $30' --limit 10 --json
scripts/search-products '100w led bulbs that cost less than $4 each and have over 200 reviews with a review score of more than 4.5 stars' --limit 5 --markdown
scripts/search-products 'USB-C cable with over 1000 reviews and rating over 4 stars' --limit 3 --json --skip-details
scripts/search-products 'sofa bed of 77 inches or wider, 4 stars or higher, 200+ reviews, shipped with Prime, delivery by tomorrow, top 10 by price' --limit 10 --json --markdown
```
Use `--dry-run` to parse a request and show planned filters without navigating to Amazon:
```bash
scripts/search-products 'USB-C charger under $30' --dry-run --json
```
Use single quotes when a request contains dollar amounts so the shell does not expand `$4`. `--max N` is accepted as a compatibility alias for `--limit N`.
## Install Or Update
Run these commands from the `stef-openclaw-skills` repo checkout:
```bash
cd ~/.openclaw/workspace/projects/stef-openclaw-skills
git pull --ff-only
rsync -a --delete \
--exclude node_modules \
--exclude dist \
--exclude coverage \
--exclude tmp \
--exclude out \
--exclude '*.log' \
--exclude '*.real.html' \
skills/amazon-shopping/ \
~/.openclaw/workspace/skills/amazon-shopping/
cd ~/.openclaw/workspace/skills/amazon-shopping
npm ci
```
Use the same sequence for a first install and for updates. `rsync --delete` keeps the active skill identical to the repo copy while preserving generated dependencies through the `node_modules` exclude. Always run `npm ci` after syncing because dependency changes are tracked through `package-lock.json`.
## Setup Verification
Verify the dependency skill, the active install, and OpenClaw discovery:
```bash
openclaw skills info web-automation
node "$HOME/.openclaw/workspace/skills/web-automation/scripts/check-install.js"
cd ~/.openclaw/workspace/skills/amazon-shopping
npm run lint
npm run typecheck
npm test
openclaw skills info amazon-shopping
```
For a quick parser-only check that does not browse Amazon:
```bash
scripts/search-products 'USB-C charger under $30' --dry-run --json
```
For a live smoke after install or update:
```bash
scripts/search-products '100w led bulbs that cost less than $4 each and have over 200 reviews with a review score of more than 4.5 stars' --limit 5 --json --markdown
```
## Dependency
This skill depends on the workspace `web-automation` skill and its CloakBrowser runtime.
## Fields
Each result includes the product ASIN, title, source URL, price, unit price when visible, rating, review count, delivery summary, specs, feature bullets, seller, availability, sponsored marker, matched filters, missing fields, and extraction notes.
Unknown or hidden fields stay unknown. The skill does not invent delivery dates, star histograms, prices, or review counts.
Markdown output uses chat-safe result blocks intended for direct user-facing answers in WhatsApp, Telegram, and terminals. Each product must keep a direct URL line:
```markdown
## Best Matches
1. Example Sofa Bed
Price: $399.99
Rating: 4.3 stars
Reviews: 250
Width: 83" OK
Prime: Prime OK
Delivery: FREE delivery Tomorrow OK
Link: https://www.amazon.com/dp/ASIN
```
The `OK` / `NO` marker is only attached to fields that correspond to requested filters. For example, `Prime OK` means the helper found a Prime signal on the search card or detail page; `not verified NO` means the product did not pass a requested Prime filter.
## Filters
Supported request filters include:
- minimum rating
- minimum review count
- maximum product price
- maximum unit price
- minimum width in inches
- Prime delivery
- delivery by today, tomorrow, or overnight
- sort by price
- result limit
- maximum search pages
`over 200 reviews` and `more than 4.5 stars` are strict comparisons. `at least 200 reviews` and `4.5 stars or better` are inclusive comparisons.
Examples of supported natural-language filters:
- `77 inches or wider`
- `shipped with Prime`
- `delivery by tomorrow`
- `overnight shipping`
- `top 10 by price`
- `rating 4.0 or better`
Equivalent CLI flags:
```bash
scripts/search-products 'sofa bed beige' --min-rating 4 --min-reviews 200 --min-width 77 --require-prime --delivery-by tomorrow --sort-by price --limit 10 --markdown
```
## Guardrails
This skill is for operator-directed product research, not purchasing automation.
- It checks Amazon robots directives before live navigation.
- It does not sign in, add to cart, purchase, access wishlists, submit reviews, crawl review pages, or bypass CAPTCHA/block pages.
- It stops and reports a warning when Amazon returns a challenge, block, or disallowed robots path.
- It uses default bounded operation: 15 results, 2 search pages, detail pages one at a time.
+297
View File
@@ -0,0 +1,297 @@
# ai-cli-dispatch Architecture
This document describes the internal design of `ai-cli-dispatch`, the module breakdown, data flow, key design decisions, and how to extend the tool.
## Module Breakdown
```text
src/
├── cli.ts — Entry point: argument parsing, command routing, I/O formatting
├── cli-helpers.ts — Shared formatting, sync/async run handlers, error reporters
├── types.ts — Shared types and error classes
├── constants.ts — Client name registry and platform helpers
├── config.ts — Layered configuration resolution (flags → env → file → PATH)
├── detect.ts — Client discovery: binary lookup and version extraction
├── dispatch.ts — Prompt-to-client resolution (explicit flag → keywords → default)
├── execute.ts — Synchronous subprocess spawning, stdout/stderr capture, timeout handling
└── jobs.ts — Async job lifecycle: detached spawn, disk-backed state, polling API
```
### Responsibilities
| Module | Responsibility |
|---|---|
| `cli.ts` | Parses `argv` with `minimist`, routes to all commands, prints JSON or text output, and controls the process exit code. |
| `cli-helpers.ts` | Shared helpers for `reportError`, `reportCliError`, `handleSyncRun`, and `handleAsyncRun` to keep `cli.ts` focused on routing. |
| `types.ts` | Defines `ClientName`, `ClientInfo`, `ExecResult`, `ToolConfig`, `Job`, `JobRecord`, `JobStatus`, and the error hierarchy (`ClientNotFoundError`, `ExecError`, `JobNotFoundError`, `JobResultUnavailableError`). |
| `constants.ts` | Holds the canonical `CLIENT_NAMES` array and `isWindows()` helper used by discovery and config. |
| `config.ts` | Resolves per-client binary paths and the optional `defaultClient` from four layered sources. |
| `detect.ts` | Locates each client binary on `PATH`, falls back to a manual directory scan, and invokes `--version` to extract a semver string. |
| `dispatch.ts` | Chooses the target client from a prompt string using ordered keyword matching, with overrides for explicit `--client` and `defaultClient`. |
| `execute.ts` | Spawns the chosen client with its native argument shape, buffers `stdout`/`stderr`, enforces a timeout, and returns an `ExecResult` or throws a typed error. |
| `jobs.ts` | Manages background jobs: writes job records to disk, spawns detached child processes, tracks running children in memory, and provides `status`, `results`, `cancel`, `list`, and `cleanup` operations. |
## Data Flow
### Synchronous dispatch (`run --sync`, `dispatch --sync`)
A sync invocation flows through four stages:
```
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ detect │ ──► │ config │ ──► │ dispatch │ ──► │ execute │
└─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘
│ │ │ │
which/where flags/env/file keyword scan spawn child
PATH walk defaultClient --client override capture output
--version fallback default timeout / exitCode
```
### Asynchronous dispatch (`run`, `dispatch`, `start`)
An async invocation adds the `jobs.ts` stage. The caller receives a job ID immediately; the child process continues in the background.
```
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ detect │ ──► │ config │ ──► │ dispatch │ ──► │ execute │ ──► │ jobs │
└─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘
│ │ │ │ │
which/where flags/env/file keyword scan spawn child write job file
PATH walk defaultClient --client override capture output detached + unref
--version fallback default timeout / exitCode update on close
```
Later, lifecycle commands read from or modify the job store:
```
status <jobId> ──► readJobFile ──► return Job (sans stdout/stderr)
results <jobId> ──► readJobFile ──► return ExecResult (completed only)
cancel <jobId> ──► readJobFile ──► kill child or PID ──► write cancelled status
list-jobs ──► readdir jobDir ──► read each file ──► sort + filter
cleanup-jobs ──► readdir jobDir ──► stat mtime ──► unlink old files
```
### 1. Detect
`detectClients()` iterates over `CLIENT_NAMES` and attempts to locate each binary:
1. Invoke `which <name>` (or `where <name>` on Windows).
2. If that fails, walk `PATH` segments manually and test `existsSync()`.
3. If a binary is found, run `<binary> --version` and parse the first semver-like match.
Result: an array of `ClientInfo` objects with `name`, `found`, `path`, and `version`.
### 2. Config
`resolveConfig()` builds a `ResolvedConfig` by layering sources (highest to lowest precedence):
1. **CLI flags**`--codex-path`, `--claude-path`, `--opencode-path`, `--default-client`, `--timeout`
2. **Environment variables**`AI_CLI_CODEX_PATH`, `AI_CLI_CLAUDE_PATH`, `AI_CLI_OPENCODE_PATH`, `AI_CLI_DEFAULT_CLIENT`
3. **Config file**`~/.openclaw/ai-cli-dispatch.json` (`paths`, `defaultClient`, `timeout` keys)
4. **PATH discovery**`which`/`where` fallback via `defaultWhichSync()`
Only values for the three known `ClientName` entries are accepted; unknown `defaultClient` values are ignored.
### 3. Dispatch
`resolveClient(prompt, config)` decides which client to use:
1. If `config.client` is a valid `ClientName`, return it immediately.
2. Lower-case the prompt and scan for substrings in order:
- `"open code"``opencode`
- `"claude"``claude`
- `"codex"``codex`
- `"opencode"``opencode`
3. If no keyword matches, return `config.defaultClient` or `null`.
This ordering intentionally prioritizes `"open code"` before `"opencode"` so the spaced natural-language variant wins.
### 4. Execute
`executePrompt(client, prompt, options)` runs the selected client synchronously:
1. Reject empty or whitespace-only prompts with `ExecError`.
2. Validate that an explicit `clientPath` exists on disk (if provided).
3. Map the client to its native argument array via `CLIENT_ARGS`:
- `codex``["exec", "--yolo", prompt]`
- `claude``["-p", prompt, "--dangerously-skip-permissions"]`
- `opencode``["run", "--dangerously-skip-permissions", prompt]`
4. `spawn()` the process with `shell: false`.
5. Buffer `stdout` and `stderr` via `"data"` listeners.
6. Start a `setTimeout`; if it fires, `child.kill()` is sent.
7. On `close`, resolve with `{ stdout, stderr, exitCode, client, durationMs }`.
8. On `error`, reject with `ClientNotFoundError` for `ENOENT` or `ExecError` for anything else.
9. On timeout, reject with `ExecError` containing the buffered output so far.
10. If `debug` is enabled, emit a `DebugInfo` object via `onDebug`.
The default timeout is **10 minutes** (`600_000` ms).
### 5. Jobs
`startJob(client, prompt, options)` launches a background job:
1. Generate a UUID for the job ID.
2. Build the client argument array via `CLIENT_ARGS`.
3. `spawn()` the process with `detached: true` and `stdio: ["ignore", "pipe", "pipe"]`.
4. Write an initial `JobRecord` to `~/.openclaw/ai-cli-dispatch/jobs/<jobId>.json` with status `running`.
5. Update the record with the child `pid` once available.
6. Register the child in an in-memory `runningChildren` Map for cancellation and timeout tracking.
7. Buffer `stdout`/`stderr` via `"data"` listeners.
8. On `close`, finalize the record: write status (`completed`, `failed`, `timed_out`, or `cancelled`), capture stdout/stderr, and record `durationMs`.
9. Call `child.unref()` so the dispatcher process can exit without waiting for the child.
`getJob(jobId)` reads the job file and returns a `Job` (omitting the full stdout/stderr buffers).
`getJobResult(jobId)` returns the `ExecResult` for a completed job.
`cancelJob(jobId)` looks up the running child in memory, sends `SIGTERM`, and writes a `cancelled` status. If the child is no longer in memory, it attempts `process.kill(pid, "SIGTERM")` as a fallback.
`listJobs({ filter })` reads all `.json` files in the job directory, parses them, sorts by `startedAt` descending, and optionally filters by status.
`cleanupJobs({ maxAgeMs })` deletes job files whose `mtime` exceeds the threshold. Default max age is 24 hours.
## Design Decisions
### Async-First Architecture
The default execution mode is **async** (background job). Synchronous execution requires an explicit `--sync` flag.
**Rationale:**
- **Primary use case alignment:** Most AI CLI tasks (refactoring, test generation, migration) run for multiple minutes. Blocking the caller for that long is often undesirable in automation and orchestration contexts.
- **Resilience:** A detached background job survives an unexpected dispatcher exit. The caller can reconnect later via `status` and `results`.
- **Batching:** Multiple jobs can be started in parallel without blocking the dispatcher process.
- **Backward compatibility path:** `--sync` preserves the original one-shot behavior for callers that need it, without changing the default.
### Disk-Backed Job Store
Job state is persisted as JSON files on disk rather than kept solely in memory.
**Rationale:**
- **Durability across restarts:** If the dispatcher process crashes or the host reboots, job files remain. A caller can still query `status` or `results` after recovery.
- **No memory leaks:** Long-running or forgotten jobs do not accumulate in heap. Cleanup is explicit via `cleanup-jobs`.
- **External observability:** Operators can inspect `~/.openclaw/ai-cli-dispatch/jobs/` directly without calling the CLI.
- **Simplicity:** A file-per-job model avoids the need for an embedded database or external service. It maps cleanly to the Node.js `fs` API and is trivial to mock in tests.
**Trade-off:** High-frequency job creation could strain the filesystem, but the expected volume is low (tens to hundreds of jobs, not thousands per second).
### Detached-Process Approach
Async jobs use `detached: true` with `child.unref()`.
**Rationale:**
- **Parent independence:** The dispatcher can start a job and exit immediately. This is essential for CLI usage where the user or orchestrator should not hold a shell open for the duration of the task.
- **Signal isolation:** A detached process group means the child does not receive `SIGINT` or `SIGHUP` sent to the dispatcher terminal session.
- **PID tracking:** Even though the child is detached, the `pid` is captured and written to the job file. This enables `cancelJob` to send signals even if the dispatcher has restarted and lost its in-memory `runningChildren` map.
**Trade-off:** The child is truly independent. If the host reboots, the child is lost (same as any other process). The job file will eventually reflect `timed_out` or remain `running` until `cancel` or `cleanup` is run.
### Coexistence with ACP
`ai-cli-dispatch` is intentionally **not** an ACP agent. It is a thin, local subprocess wrapper with no session state, no thread binding, and no orchestrator protocol.
- Use `ai-cli-dispatch` when you need a quick, one-shot CLI execution or a background job on the gateway host.
- Use ACP (`docs/openclaw-acp-orchestration.md`) when you need session-bound coding harnesses, multi-turn review, or orchestrator-managed verification gates.
This separation keeps the dispatcher small and avoids duplicating ACPs scheduling, context persistence, and review-loop responsibilities.
### Keyword Dispatch vs NLP
Client resolution uses deterministic substring matching instead of natural-language parsing or an LLM call.
**Rationale:**
- **Speed:** No network round-trip or model load; resolution is synchronous and sub-millisecond.
- **Predictability:** The same prompt always resolves to the same client. There is no temperature, context window, or model-version drift.
- **Debuggability:** A user can read the ordered keyword list and know exactly why a given prompt resolved to a given client.
- **Scope fit:** The dispatcher only needs to distinguish three clients. A full NLP pipeline would be overkill.
The trade-off is that prompts like `"compare codex and claude"` resolve to `codex` because `"codex"` is checked first. Users can always override with `--client`.
### Error Taxonomy
All runtime failures are represented as typed errors so callers and tests can branch precisely:
| Error | When thrown | Data carried |
|---|---|---|
| `ClientNotFoundError` | Binary not on `PATH`, explicit `clientPath` missing, or `ENOENT` from `spawn` | `message` with client name |
| `ExecError` | Empty prompt, unknown client, timeout, non-`ENOENT` spawn error, or child exit | `message` + full `ExecResult` (`stdout`, `stderr`, `exitCode`, `client`, `durationMs`) |
| `JobNotFoundError` | Job ID not found in the job store | `message` with job ID |
| `JobResultUnavailableError` | `results` called on a non-completed job | `message` with job ID and current status |
`ExecError` carries the `ExecResult` so that timeout handlers still return partial output. This avoids losing buffered stdout/stderr when a long-running task is killed.
### Injection-Friendly Module Boundaries
Every non-trivial module accepts an `options` bag with injectable dependencies (`spawnSync`, `spawn`, `existsSync`, `whichSync`, `readFileSync`, etc.).
**Rationale:**
- Unit tests can run without touching the real filesystem, `PATH`, or subprocess layer.
- The CLI itself injects its real dependencies through default parameters, so production behavior is unchanged.
- There is no global mocking required; each test provides its own narrow fakes.
### Minimal Dependency Surface
The runtime dependency graph contains exactly one external package: `minimist` (argument parsing). Everything else uses Node.js built-ins (`child_process`, `fs`, `os`, `path`, `crypto`).
**Rationale:**
- Reduces supply-chain risk and install time.
- Avoids version-lock issues across Node.js 20+ environments.
- Keeps the compiled/bundled footprint negligible for a tool that is often installed as a sidecar.
## Extension Points
### Adding a New Client
To support a fourth (or fifth) AI CLI client, change four files in `src/` and the corresponding tests:
1. **`src/types.ts`** — Add the new name to the `ClientName` union type.
2. **`src/constants.ts`** — Append the new name to `CLIENT_NAMES`.
3. **`src/execute.ts`** — Add an entry to `CLIENT_ARGS` with the clients native argument shape.
4. **`src/config.ts`** — No change required; the existing loop over `CLIENT_NAMES` automatically picks up the new env/flag/file keys.
5. **`src/dispatch.ts`** — Add a keyword check for the new client in `resolveClient`. Decide its precedence relative to existing keywords.
6. **`src/jobs.ts`** — No change required; `CLIENT_ARGS` is already shared.
7. **Tests** — Add colocated test cases in `tests/dispatch.test.ts`, `tests/execute.test.ts`, `tests/detect.test.ts`, and `tests/jobs.test.ts`.
No changes are needed in `cli.ts` because it iterates over `CLIENT_NAMES` for validation.
### Streaming Support
If a future use case requires real-time output (e.g., long-running codegen with progressive feedback), the cleanest extension is to add an optional `onData` callback to `ExecuteOptions`:
```typescript
export interface ExecuteOptions {
clientPath?: string;
timeoutMs?: number;
spawn?: ...;
existsSync?: ...;
onData?: (chunk: string, stream: "stdout" | "stderr") => void;
}
```
When `onData` is provided, `executePrompt` would:
- Continue buffering internally for the final `ExecResult`.
- Also emit each chunk through `onData` so the caller can stream to a UI or logger.
- Reject/resolve with the same error taxonomy.
This preserves backward compatibility: existing callers that omit `onData` receive the exact same buffered `ExecResult` they get today.
For async jobs, `jobs.ts` could store a partial `stdout`/`stderr` in the job file on each chunk (or at a throttled interval) so `status` callers can see progress without waiting for completion.
### Platform Backends
The current Windows support is limited to discovery (`where` instead of `which`, `.exe` extension assumptions). If future clients require platform-specific spawn options (e.g., PowerShell quoting rules), the extension point is `CLIENT_ARGS` or a new `CLIENT_SPAWN_OPTIONS` record keyed by `ClientName`.
## Testing Strategy
The test suite in `tests/` mirrors the `src/` structure:
| Test file | Coverage |
|---|---|
| `cli.test.ts` | Argument parsing, command routing, JSON/text output modes, exit codes, error formatting, sync vs async branches, all job lifecycle commands |
| `cli-helpers.test.ts` | `reportError`, `reportCliError`, `handleSyncRun`, `handleAsyncRun` with JSON and text modes |
| `config.test.ts` | Layered precedence of flags, env, file, and `which` fallback; malformed JSON tolerance |
| `detect.test.ts` | `which` success/failure, PATH directory fallback, version parsing, missing binary handling |
| `dispatch.test.ts` | Keyword matching, case insensitivity, `--client` precedence, `defaultClient` fallback, invalid flag handling |
| `execute.test.ts` | Successful execution, stderr capture, non-zero exit codes, `ENOENT``ClientNotFoundError`, timeout, empty prompt rejection, special-character preservation, debug info emission |
| `jobs.test.ts` | Job start, status query, result retrieval, cancellation, listing, cleanup, timeout handling, unknown client fallback, detached process behavior, in-memory vs on-disk consistency |
All tests use injected mocks; no test spawns real client binaries or reads the real filesystem.
+219
View File
@@ -0,0 +1,219 @@
# Installation
This page covers installing the `ai-cli-dispatch` tool, its prerequisites, and post-install verification.
## Prerequisites
- **Node.js** ≥ 20 (required for `tsx`, `import` attributes, and modern `node:child_process` APIs)
- **npm** (bundled with Node.js)
- **Homebrew** (macOS/Linux) — recommended for installing the underlying AI CLI clients (`codex`, `claude`, `opencode`)
- One or more supported AI CLI clients:
- [Codex CLI](https://github.com/openai/codex)
- [Claude Code](https://docs.anthropic.com/en/docs/agents-and-tools/claude-code/overview)
- [OpenCode](https://github.com/opencode-ai/opencode)
## Install the tool
### 1. Clone the repository
```bash
git clone <repository-url> ~/.openclaw/workspace/skills/ai-cli-dispatch
cd ~/.openclaw/workspace/skills/ai-cli-dispatch
```
If you already have the full `stef-openclaw-skills` repo checked out, use the path inside it instead:
```bash
cd ~/.openclaw/workspace/skills/stef-openclaw-skills/tools/ai-cli-dispatch
```
### 2. Install Node dependencies
```bash
npm install
```
This installs:
- `tsx` — TypeScript execution runtime
- `minimist` — argument parsing
- `typescript` — type checking
## PATH configuration
The helper script lives at `scripts/ai-cli-dispatch`. Add it to your shell PATH so OpenClaw (or your terminal) can invoke it without a full path:
```bash
# ~/.zshrc or ~/.bashrc
export PATH="$HOME/.openclaw/workspace/skills/ai-cli-dispatch/scripts:$PATH"
```
Reload your shell:
```bash
source ~/.zshrc # or ~/.bashrc
```
Verify the script is reachable:
```bash
which ai-cli-dispatch
ai-cli-dispatch --help
```
## Optional configuration file
Create `~/.openclaw/ai-cli-dispatch.json` to customize client paths and set a default client:
```bash
mkdir -p ~/.openclaw
$EDITOR ~/.openclaw/ai-cli-dispatch.json
```
Example configuration:
```json
{
"paths": {
"codex": "/opt/homebrew/bin/codex",
"claude": "/opt/homebrew/bin/claude",
"opencode": "/opt/homebrew/bin/opencode"
},
"defaultClient": "claude"
}
```
### Configuration precedence
When resolving a client binary, the tool checks sources in this order (first match wins):
1. CLI flag: `--codex-path`, `--claude-path`, `--opencode-path`
2. Environment variable: `AI_CLI_CODEX_PATH`, `AI_CLI_CLAUDE_PATH`, `AI_CLI_OPENCODE_PATH`
3. File config: `paths.<client>` in `~/.openclaw/ai-cli-dispatch.json`
4. System `PATH` lookup via `which` / `where`
The `defaultClient` follows the same precedence:
1. CLI flag: `--default-client`
2. Environment variable: `AI_CLI_DEFAULT_CLIENT`
3. File config: `defaultClient` in `~/.openclaw/ai-cli-dispatch.json`
## Install AI CLI clients
### Codex
```bash
npm install -g @openai/codex
```
### Claude Code
```bash
npm install -g @anthropic-ai/claude-code
```
### OpenCode
```bash
npm install -g @opencode-ai/opencode
```
Or via Homebrew where formulas are available:
```bash
brew install codex # if available in your tap
brew install claude-code # if available in your tap
```
## Verification
### 1. Check local tool health
```bash
ai-cli-dispatch --help
```
Expected output:
```text
AI CLI Dispatch
Usage:
ai-cli-dispatch list [--json|--text]
ai-cli-dispatch run --client <client> --prompt <prompt> [--json|--text]
ai-cli-dispatch dispatch <prompt> [--client <client>] [--json|--text]
ai-cli-dispatch --help
Clients: codex, claude, opencode
```
### 2. List discovered clients
```bash
ai-cli-dispatch list --json
```
Example output when two clients are installed:
```json
[
{
"name": "codex",
"path": "/opt/homebrew/bin/codex",
"version": "1.2.3",
"found": true
},
{
"name": "claude",
"path": "/opt/homebrew/bin/claude",
"version": "0.7.8",
"found": true
},
{
"name": "opencode",
"found": false
}
]
```
### 3. Run a quick dispatch
```bash
ai-cli-dispatch run --client codex --prompt "hello" --json
```
This should return a JSON result with `stdout`, `stderr`, and `exitCode`.
### 4. Test keyword dispatch
```bash
ai-cli-dispatch dispatch "refactor this using claude"
```
The tool inspects the prompt for client keywords (`claude`, `codex`, `opencode`, `open code`) and routes to the matching client.
## Troubleshooting
### `Missing local Node dependencies for ai-cli-dispatch`
Run `npm install` from the skill directory:
```bash
cd ~/.openclaw/workspace/skills/ai-cli-dispatch
npm install
```
### `Client "codex" not found or not installed`
- Ensure the client is installed globally or via Homebrew
- Verify it is on your PATH: `which codex`
- Or override the path in `~/.openclaw/ai-cli-dispatch.json` or with an environment variable
### `Prompt cannot be empty`
The `run` and `dispatch` commands require a non-empty `--prompt` or trailing prompt text.
### Config file is not being read
- Verify the file is at exactly `~/.openclaw/ai-cli-dispatch.json`
- Check for JSON syntax errors (trailing commas are not allowed)
- Use `--debug` for deeper troubleshooting if supported by the calling context
+7
View File
@@ -0,0 +1,7 @@
node_modules/
dist/
coverage/
*.log
tmp/
out/
*.real.html
+42
View File
@@ -0,0 +1,42 @@
---
name: amazon-shopping
description: Search amazon.com shopping results with product filters using the local web-automation skill. Use when the user asks to find, compare, filter, or summarize Amazon products by description, price, delivery, specs, review count, star rating, or star distribution.
---
# Amazon Shopping
Use this skill for read-only Amazon product discovery and comparison.
## First Checks
Verify the browser dependency before live use:
```bash
openclaw skills info web-automation
node "$HOME/.openclaw/workspace/skills/web-automation/scripts/check-install.js"
```
## Search Products
Run the helper from the installed skill directory:
```bash
"$HOME/.openclaw/workspace/skills/amazon-shopping/scripts/search-products" '<user request>' --json --markdown
```
Use single quotes around product requests that contain dollar amounts so the shell does not expand `$4` or similar text. Use `--limit N`; `--max N` is accepted as a compatibility alias. If your execution tool supports a timeout option, set it to at least 600 seconds for live runs with detail enrichment. Use `--skip-details` only for a quick preview or when the user does not need specifications and delivery details.
Default to at most 15 products unless the user asks for a different count. For requested counts above 30, ask before continuing or split the request into batches. Always include source URLs, report missing fields explicitly, and do not claim review histogram data unless it was visible and extracted.
For user-facing answers, use the generated chat-safe result blocks as the presentation template. Keep the direct `Link: https://...` line for every product because WhatsApp and Telegram do not reliably render markdown tables. Do not rewrite Prime or delivery status as verified unless the helper marks it verified.
Supported filters include minimum rating, minimum reviews, maximum price, maximum unit price, minimum width in inches, Prime delivery, delivery by today/tomorrow/overnight, and sort by price. Natural language such as `77 inches or wider`, `shipped with Prime`, `delivery by tomorrow`, `overnight shipping`, and `top 10 by price` is parsed automatically. CLI flags are also available: `--min-width`, `--require-prime`, `--delivery-by`, and `--sort-by price`.
## Guardrails
This skill is for operator-directed, read-only product research. Before live scraping, the helper checks Amazon robots directives for planned paths. Do not automate sign-in, cart, purchase, wishlist, review submission, review-page crawling, CAPTCHA bypass, or blocked-page bypass. If Amazon returns a challenge or block page, stop and report that status.
Read references when needed:
- `references/amazon-data-map.md` for fields and selectors.
- `references/web-automation-prompts.md` for browser extraction prompts.
- `references/compliance-and-failure-modes.md` for blocked-page and unknown-field behavior.
+743
View File
@@ -0,0 +1,743 @@
{
"name": "amazon-shopping-scripts",
"version": "1.0.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "amazon-shopping-scripts",
"version": "1.0.0",
"dependencies": {
"minimist": "^1.2.8",
"node-html-parser": "^7.1.0"
},
"devDependencies": {
"@types/minimist": "^1.2.5",
"@types/node": "^24.8.1",
"tsx": "^4.20.6",
"typescript": "^5.9.3"
}
},
"node_modules/@esbuild/aix-ppc64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.27.7.tgz",
"integrity": "sha512-EKX3Qwmhz1eMdEJokhALr0YiD0lhQNwDqkPYyPhiSwKrh7/4KRjQc04sZ8db+5DVVnZ1LmbNDI1uAMPEUBnQPg==",
"cpu": [
"ppc64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"aix"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-arm": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.27.7.tgz",
"integrity": "sha512-jbPXvB4Yj2yBV7HUfE2KHe4GJX51QplCN1pGbYjvsyCZbQmies29EoJbkEc+vYuU5o45AfQn37vZlyXy4YJ8RQ==",
"cpu": [
"arm"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-arm64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.27.7.tgz",
"integrity": "sha512-62dPZHpIXzvChfvfLJow3q5dDtiNMkwiRzPylSCfriLvZeq0a1bWChrGx/BbUbPwOrsWKMn8idSllklzBy+dgQ==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/android-x64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.27.7.tgz",
"integrity": "sha512-x5VpMODneVDb70PYV2VQOmIUUiBtY3D3mPBG8NxVk5CogneYhkR7MmM3yR/uMdITLrC1ml/NV1rj4bMJuy9MCg==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"android"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/darwin-arm64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.27.7.tgz",
"integrity": "sha512-5lckdqeuBPlKUwvoCXIgI2D9/ABmPq3Rdp7IfL70393YgaASt7tbju3Ac+ePVi3KDH6N2RqePfHnXkaDtY9fkw==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/darwin-x64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.27.7.tgz",
"integrity": "sha512-rYnXrKcXuT7Z+WL5K980jVFdvVKhCHhUwid+dDYQpH+qu+TefcomiMAJpIiC2EM3Rjtq0sO3StMV/+3w3MyyqQ==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/freebsd-arm64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.27.7.tgz",
"integrity": "sha512-B48PqeCsEgOtzME2GbNM2roU29AMTuOIN91dsMO30t+Ydis3z/3Ngoj5hhnsOSSwNzS+6JppqWsuhTp6E82l2w==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/freebsd-x64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.27.7.tgz",
"integrity": "sha512-jOBDK5XEjA4m5IJK3bpAQF9/Lelu/Z9ZcdhTRLf4cajlB+8VEhFFRjWgfy3M1O4rO2GQ/b2dLwCUGpiF/eATNQ==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-arm": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.27.7.tgz",
"integrity": "sha512-RkT/YXYBTSULo3+af8Ib0ykH8u2MBh57o7q/DAs3lTJlyVQkgQvlrPTnjIzzRPQyavxtPtfg0EopvDyIt0j1rA==",
"cpu": [
"arm"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-arm64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.27.7.tgz",
"integrity": "sha512-RZPHBoxXuNnPQO9rvjh5jdkRmVizktkT7TCDkDmQ0W2SwHInKCAV95GRuvdSvA7w4VMwfCjUiPwDi0ZO6Nfe9A==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-ia32": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.27.7.tgz",
"integrity": "sha512-GA48aKNkyQDbd3KtkplYWT102C5sn/EZTY4XROkxONgruHPU72l+gW+FfF8tf2cFjeHaRbWpOYa/uRBz/Xq1Pg==",
"cpu": [
"ia32"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-loong64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.27.7.tgz",
"integrity": "sha512-a4POruNM2oWsD4WKvBSEKGIiWQF8fZOAsycHOt6JBpZ+JN2n2JH9WAv56SOyu9X5IqAjqSIPTaJkqN8F7XOQ5Q==",
"cpu": [
"loong64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-mips64el": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.27.7.tgz",
"integrity": "sha512-KabT5I6StirGfIz0FMgl1I+R1H73Gp0ofL9A3nG3i/cYFJzKHhouBV5VWK1CSgKvVaG4q1RNpCTR2LuTVB3fIw==",
"cpu": [
"mips64el"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-ppc64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.27.7.tgz",
"integrity": "sha512-gRsL4x6wsGHGRqhtI+ifpN/vpOFTQtnbsupUF5R5YTAg+y/lKelYR1hXbnBdzDjGbMYjVJLJTd2OFmMewAgwlQ==",
"cpu": [
"ppc64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-riscv64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.27.7.tgz",
"integrity": "sha512-hL25LbxO1QOngGzu2U5xeXtxXcW+/GvMN3ejANqXkxZ/opySAZMrc+9LY/WyjAan41unrR3YrmtTsUpwT66InQ==",
"cpu": [
"riscv64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-s390x": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.27.7.tgz",
"integrity": "sha512-2k8go8Ycu1Kb46vEelhu1vqEP+UeRVj2zY1pSuPdgvbd5ykAw82Lrro28vXUrRmzEsUV0NzCf54yARIK8r0fdw==",
"cpu": [
"s390x"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/linux-x64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.27.7.tgz",
"integrity": "sha512-hzznmADPt+OmsYzw1EE33ccA+HPdIqiCRq7cQeL1Jlq2gb1+OyWBkMCrYGBJ+sxVzve2ZJEVeePbLM2iEIZSxA==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/netbsd-arm64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.27.7.tgz",
"integrity": "sha512-b6pqtrQdigZBwZxAn1UpazEisvwaIDvdbMbmrly7cDTMFnw/+3lVxxCTGOrkPVnsYIosJJXAsILG9XcQS+Yu6w==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/netbsd-x64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.27.7.tgz",
"integrity": "sha512-OfatkLojr6U+WN5EDYuoQhtM+1xco+/6FSzJJnuWiUw5eVcicbyK3dq5EeV/QHT1uy6GoDhGbFpprUiHUYggrw==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openbsd-arm64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.27.7.tgz",
"integrity": "sha512-AFuojMQTxAz75Fo8idVcqoQWEHIXFRbOc1TrVcFSgCZtQfSdc1RXgB3tjOn/krRHENUB4j00bfGjyl2mJrU37A==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openbsd-x64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.27.7.tgz",
"integrity": "sha512-+A1NJmfM8WNDv5CLVQYJ5PshuRm/4cI6WMZRg1by1GwPIQPCTs1GLEUHwiiQGT5zDdyLiRM/l1G0Pv54gvtKIg==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/openharmony-arm64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.27.7.tgz",
"integrity": "sha512-+KrvYb/C8zA9CU/g0sR6w2RBw7IGc5J2BPnc3dYc5VJxHCSF1yNMxTV5LQ7GuKteQXZtspjFbiuW5/dOj7H4Yw==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"openharmony"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/sunos-x64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.27.7.tgz",
"integrity": "sha512-ikktIhFBzQNt/QDyOL580ti9+5mL/YZeUPKU2ivGtGjdTYoqz6jObj6nOMfhASpS4GU4Q/Clh1QtxWAvcYKamA==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"sunos"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-arm64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.27.7.tgz",
"integrity": "sha512-7yRhbHvPqSpRUV7Q20VuDwbjW5kIMwTHpptuUzV+AA46kiPze5Z7qgt6CLCK3pWFrHeNfDd1VKgyP4O+ng17CA==",
"cpu": [
"arm64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-ia32": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.27.7.tgz",
"integrity": "sha512-SmwKXe6VHIyZYbBLJrhOoCJRB/Z1tckzmgTLfFYOfpMAx63BJEaL9ExI8x7v0oAO3Zh6D/Oi1gVxEYr5oUCFhw==",
"cpu": [
"ia32"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@esbuild/win32-x64": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.27.7.tgz",
"integrity": "sha512-56hiAJPhwQ1R4i+21FVF7V8kSD5zZTdHcVuRFMW0hn753vVfQN8xlx4uOPT4xoGH0Z/oVATuR82AiqSTDIpaHg==",
"cpu": [
"x64"
],
"dev": true,
"license": "MIT",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=18"
}
},
"node_modules/@types/minimist": {
"version": "1.2.5",
"resolved": "https://registry.npmjs.org/@types/minimist/-/minimist-1.2.5.tgz",
"integrity": "sha512-hov8bUuiLiyFPGyFPE1lwWhmzYbirOXQNNo40+y3zow8aFVTeyn3VWL0VFFfdNddA8S4Vf0Tc062rzyNr7Paag==",
"dev": true,
"license": "MIT"
},
"node_modules/@types/node": {
"version": "24.12.2",
"resolved": "https://registry.npmjs.org/@types/node/-/node-24.12.2.tgz",
"integrity": "sha512-A1sre26ke7HDIuY/M23nd9gfB+nrmhtYyMINbjI1zHJxYteKR6qSMX56FsmjMcDb3SMcjJg5BiRRgOCC/yBD0g==",
"dev": true,
"license": "MIT",
"dependencies": {
"undici-types": "~7.16.0"
}
},
"node_modules/boolbase": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/boolbase/-/boolbase-1.0.0.tgz",
"integrity": "sha512-JZOSA7Mo9sNGB8+UjSgzdLtokWAky1zbztM3WRLCbZ70/3cTANmQmOdR7y2g+J0e2WXywy1yS468tY+IruqEww==",
"license": "ISC"
},
"node_modules/css-select": {
"version": "5.2.2",
"resolved": "https://registry.npmjs.org/css-select/-/css-select-5.2.2.tgz",
"integrity": "sha512-TizTzUddG/xYLA3NXodFM0fSbNizXjOKhqiQQwvhlspadZokn1KDy0NZFS0wuEubIYAV5/c1/lAr0TaaFXEXzw==",
"license": "BSD-2-Clause",
"dependencies": {
"boolbase": "^1.0.0",
"css-what": "^6.1.0",
"domhandler": "^5.0.2",
"domutils": "^3.0.1",
"nth-check": "^2.0.1"
},
"funding": {
"url": "https://github.com/sponsors/fb55"
}
},
"node_modules/css-what": {
"version": "6.2.2",
"resolved": "https://registry.npmjs.org/css-what/-/css-what-6.2.2.tgz",
"integrity": "sha512-u/O3vwbptzhMs3L1fQE82ZSLHQQfto5gyZzwteVIEyeaY5Fc7R4dapF/BvRoSYFeqfBk4m0V1Vafq5Pjv25wvA==",
"license": "BSD-2-Clause",
"engines": {
"node": ">= 6"
},
"funding": {
"url": "https://github.com/sponsors/fb55"
}
},
"node_modules/dom-serializer": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/dom-serializer/-/dom-serializer-2.0.0.tgz",
"integrity": "sha512-wIkAryiqt/nV5EQKqQpo3SToSOV9J0DnbJqwK7Wv/Trc92zIAYZ4FlMu+JPFW1DfGFt81ZTCGgDEabffXeLyJg==",
"license": "MIT",
"dependencies": {
"domelementtype": "^2.3.0",
"domhandler": "^5.0.2",
"entities": "^4.2.0"
},
"funding": {
"url": "https://github.com/cheeriojs/dom-serializer?sponsor=1"
}
},
"node_modules/domelementtype": {
"version": "2.3.0",
"resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-2.3.0.tgz",
"integrity": "sha512-OLETBj6w0OsagBwdXnPdN0cnMfF9opN69co+7ZrbfPGrdpPVNBUj02spi6B1N7wChLQiPn4CSH/zJvXw56gmHw==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/fb55"
}
],
"license": "BSD-2-Clause"
},
"node_modules/domhandler": {
"version": "5.0.3",
"resolved": "https://registry.npmjs.org/domhandler/-/domhandler-5.0.3.tgz",
"integrity": "sha512-cgwlv/1iFQiFnU96XXgROh8xTeetsnJiDsTc7TYCLFd9+/WNkIqPTxiM/8pSd8VIrhXGTf1Ny1q1hquVqDJB5w==",
"license": "BSD-2-Clause",
"dependencies": {
"domelementtype": "^2.3.0"
},
"engines": {
"node": ">= 4"
},
"funding": {
"url": "https://github.com/fb55/domhandler?sponsor=1"
}
},
"node_modules/domutils": {
"version": "3.2.2",
"resolved": "https://registry.npmjs.org/domutils/-/domutils-3.2.2.tgz",
"integrity": "sha512-6kZKyUajlDuqlHKVX1w7gyslj9MPIXzIFiz/rGu35uC1wMi+kMhQwGhl4lt9unC9Vb9INnY9Z3/ZA3+FhASLaw==",
"license": "BSD-2-Clause",
"dependencies": {
"dom-serializer": "^2.0.0",
"domelementtype": "^2.3.0",
"domhandler": "^5.0.3"
},
"funding": {
"url": "https://github.com/fb55/domutils?sponsor=1"
}
},
"node_modules/entities": {
"version": "4.5.0",
"resolved": "https://registry.npmjs.org/entities/-/entities-4.5.0.tgz",
"integrity": "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw==",
"license": "BSD-2-Clause",
"engines": {
"node": ">=0.12"
},
"funding": {
"url": "https://github.com/fb55/entities?sponsor=1"
}
},
"node_modules/esbuild": {
"version": "0.27.7",
"resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.27.7.tgz",
"integrity": "sha512-IxpibTjyVnmrIQo5aqNpCgoACA/dTKLTlhMHihVHhdkxKyPO1uBBthumT0rdHmcsk9uMonIWS0m4FljWzILh3w==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
"bin": {
"esbuild": "bin/esbuild"
},
"engines": {
"node": ">=18"
},
"optionalDependencies": {
"@esbuild/aix-ppc64": "0.27.7",
"@esbuild/android-arm": "0.27.7",
"@esbuild/android-arm64": "0.27.7",
"@esbuild/android-x64": "0.27.7",
"@esbuild/darwin-arm64": "0.27.7",
"@esbuild/darwin-x64": "0.27.7",
"@esbuild/freebsd-arm64": "0.27.7",
"@esbuild/freebsd-x64": "0.27.7",
"@esbuild/linux-arm": "0.27.7",
"@esbuild/linux-arm64": "0.27.7",
"@esbuild/linux-ia32": "0.27.7",
"@esbuild/linux-loong64": "0.27.7",
"@esbuild/linux-mips64el": "0.27.7",
"@esbuild/linux-ppc64": "0.27.7",
"@esbuild/linux-riscv64": "0.27.7",
"@esbuild/linux-s390x": "0.27.7",
"@esbuild/linux-x64": "0.27.7",
"@esbuild/netbsd-arm64": "0.27.7",
"@esbuild/netbsd-x64": "0.27.7",
"@esbuild/openbsd-arm64": "0.27.7",
"@esbuild/openbsd-x64": "0.27.7",
"@esbuild/openharmony-arm64": "0.27.7",
"@esbuild/sunos-x64": "0.27.7",
"@esbuild/win32-arm64": "0.27.7",
"@esbuild/win32-ia32": "0.27.7",
"@esbuild/win32-x64": "0.27.7"
}
},
"node_modules/fsevents": {
"version": "2.3.3",
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz",
"integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==",
"dev": true,
"hasInstallScript": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
}
},
"node_modules/get-tsconfig": {
"version": "4.14.0",
"resolved": "https://registry.npmjs.org/get-tsconfig/-/get-tsconfig-4.14.0.tgz",
"integrity": "sha512-yTb+8DXzDREzgvYmh6s9vHsSVCHeC0G3PI5bEXNBHtmshPnO+S5O7qgLEOn0I5QvMy6kpZN8K1NKGyilLb93wA==",
"dev": true,
"license": "MIT",
"dependencies": {
"resolve-pkg-maps": "^1.0.0"
},
"funding": {
"url": "https://github.com/privatenumber/get-tsconfig?sponsor=1"
}
},
"node_modules/he": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/he/-/he-1.2.0.tgz",
"integrity": "sha512-F/1DnUGPopORZi0ni+CvrCgHQ5FyEAHRLSApuYWMmrbSwoN2Mn/7k+Gl38gJnR7yyDZk6WLXwiGod1JOWNDKGw==",
"license": "MIT",
"bin": {
"he": "bin/he"
}
},
"node_modules/minimist": {
"version": "1.2.8",
"resolved": "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz",
"integrity": "sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA==",
"license": "MIT",
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/node-html-parser": {
"version": "7.1.0",
"resolved": "https://registry.npmjs.org/node-html-parser/-/node-html-parser-7.1.0.tgz",
"integrity": "sha512-iJo8b2uYGT40Y8BTyy5ufL6IVbN8rbm/1QK2xffXU/1a/v3AAa0d1YAoqBNYqaS4R/HajkWIpIfdE6KcyFh1AQ==",
"license": "MIT",
"dependencies": {
"css-select": "^5.1.0",
"he": "1.2.0"
}
},
"node_modules/nth-check": {
"version": "2.1.1",
"resolved": "https://registry.npmjs.org/nth-check/-/nth-check-2.1.1.tgz",
"integrity": "sha512-lqjrjmaOoAnWfMmBPL+XNnynZh2+swxiX3WUE0s4yEHI6m+AwrK2UZOimIRl3X/4QctVqS8AiZjFqyOGrMXb/w==",
"license": "BSD-2-Clause",
"dependencies": {
"boolbase": "^1.0.0"
},
"funding": {
"url": "https://github.com/fb55/nth-check?sponsor=1"
}
},
"node_modules/resolve-pkg-maps": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/resolve-pkg-maps/-/resolve-pkg-maps-1.0.0.tgz",
"integrity": "sha512-seS2Tj26TBVOC2NIc2rOe2y2ZO7efxITtLZcGSOnHHNOQ7CkiUBfw0Iw2ck6xkIhPwLhKNLS8BO+hEpngQlqzw==",
"dev": true,
"license": "MIT",
"funding": {
"url": "https://github.com/privatenumber/resolve-pkg-maps?sponsor=1"
}
},
"node_modules/tsx": {
"version": "4.21.0",
"resolved": "https://registry.npmjs.org/tsx/-/tsx-4.21.0.tgz",
"integrity": "sha512-5C1sg4USs1lfG0GFb2RLXsdpXqBSEhAaA/0kPL01wxzpMqLILNxIxIOKiILz+cdg/pLnOUxFYOR5yhHU666wbw==",
"dev": true,
"license": "MIT",
"dependencies": {
"esbuild": "~0.27.0",
"get-tsconfig": "^4.7.5"
},
"bin": {
"tsx": "dist/cli.mjs"
},
"engines": {
"node": ">=18.0.0"
},
"optionalDependencies": {
"fsevents": "~2.3.3"
}
},
"node_modules/typescript": {
"version": "5.9.3",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz",
"integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==",
"dev": true,
"license": "Apache-2.0",
"bin": {
"tsc": "bin/tsc",
"tsserver": "bin/tsserver"
},
"engines": {
"node": ">=14.17"
}
},
"node_modules/undici-types": {
"version": "7.16.0",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-7.16.0.tgz",
"integrity": "sha512-Zz+aZWSj8LE6zoxD+xrjh4VfkIG8Ya6LvYkZqtUQGJPZjYl53ypCaUwWqo7eI0x66KBGeRo+mlBEkMSeSZ38Nw==",
"dev": true,
"license": "MIT"
}
}
}
+22
View File
@@ -0,0 +1,22 @@
{
"name": "amazon-shopping-scripts",
"version": "1.0.0",
"description": "Amazon shopping helper CLI for OpenClaw skills",
"type": "module",
"scripts": {
"amazon-shopping": "tsx src/cli.ts",
"lint": "node scripts/lint.mjs",
"test": "node --import tsx --test tests/*.test.ts",
"typecheck": "tsc --noEmit"
},
"dependencies": {
"minimist": "^1.2.8",
"node-html-parser": "^7.1.0"
},
"devDependencies": {
"@types/minimist": "^1.2.5",
"@types/node": "^24.8.1",
"tsx": "^4.20.6",
"typescript": "^5.9.3"
}
}
@@ -0,0 +1,52 @@
# Amazon Data Map
Use this reference when deciding which visible Amazon fields can be reported by `amazon-shopping`.
## Product Search Fields
Search result cards should be treated as candidates, not final truth. Prefer cards with a non-empty `data-asin` value. Extract only visible data from the rendered search page:
| Output field | Search-page source | Notes |
|---|---|---|
| `asin` | `data-asin` on result card | Required for normalized detail links. |
| `title` | product heading or product link text | Trim sponsored/accessibility boilerplate. |
| `url` | product link | Normalize to `https://www.amazon.com/dp/<ASIN>` when safe. |
| `imageUrl` | visible product image `src` | Optional. |
| `price` | visible `.a-price` text | Do not infer absent prices from snippets. |
| `rating` | `aria-label` or visible text like `4.6 out of 5 stars` | Optional. |
| `reviewCount` | text near rating like `1,234 ratings` | Optional. |
| `delivery.display` | visible delivery promise text | Optional and ZIP/session dependent. |
| `delivery.prime` | visible Prime badge, Prime icon class, `aria-label`, `alt`, or delivery text | Optional and ZIP/session dependent. Preserve a true search-card Prime signal when detail text omits the literal word Prime. |
| `isSponsored` | visible sponsored marker | Sponsored results may be included but must be labeled. |
## Detail Page Fields
Open only normalized product detail URLs under `/dp/<ASIN>` or `/gp/product/<ASIN>`. Extract visible fields:
| Output field | Detail-page source | Notes |
|---|---|---|
| `title` | `#productTitle` or equivalent heading | Detail title can replace search title. |
| `price` | buy-box/current price selectors | Variant pages can omit price. |
| `delivery` | delivery message near buy box | Report as text, not guaranteed. |
| `availability` | availability block | Optional. |
| `seller` | seller/ships-from visible text | Optional. |
| `bullets` | feature bullets list | Trim empty and hidden items. |
| `specs` | product overview/details/technical tables | Preserve name/value pairs. |
| `starBreakdown` | visible customer-review histogram | Percent or count basis only. Do not crawl review pages. |
## Filter Semantics
- `over 200 reviews` means `reviewCount > 200`.
- `at least 200 reviews` means `reviewCount >= 200`.
- `more than 4.5 stars` means `rating > 4.5`.
- `4.5 stars or better` means `rating >= 4.5`.
- `less than $4 each` means visible unit price first, then high-confidence unit-count inference. Unknown unit prices do not pass strict unit-price filters.
- `77 inches or wider` means the overall product width must be `>= 77` inches. Prefer product/item dimensions with an explicit `W` component; ignore seat, arm, door, package, and cushion widths.
- `shipped with Prime` / `Prime shipping` means a visible Prime signal must be detected on the search card or detail page.
- `delivery by tomorrow` and `overnight shipping` require visible delivery text that indicates tomorrow, overnight, next-day, or one-day delivery.
- `top 10 by price` sorts passing products by displayed product price ascending.
- Missing fields must be represented as `null` or noted in `missingFields` / `extractionNotes`; never fabricate values.
## Official Alternatives
Amazon Business Product Search API and Product Advertising API are official API paths for structured product data when the operator has credentials. This skill uses bounded web automation because the current install request requires `web-automation` scraping.
@@ -0,0 +1,39 @@
# Compliance And Failure Modes
This reference is operational guidance, not legal advice. The operator is responsible for making sure a run complies with Amazon terms, robots directives, local law, and account obligations.
## Required Guardrails
- Fetch and evaluate `https://www.amazon.com/robots.txt` before live scraping planned Amazon paths.
- Stop if the effective rules disallow the planned search or detail paths.
- Do not automate sign-in, checkout, cart, wishlist, review submission, customer-review pages, reviewer profiles, or any disallowed path.
- Do not bypass CAPTCHA, bot checks, blocked pages, or access-denied pages.
- Do not print cookies, profile state, session storage, or account/location-specific browser data.
## Allowed Scope
Allowed behavior is bounded read-only product research over search result pages and normalized product detail pages:
- `/s?k=<query>` search results.
- `/dp/<ASIN>` product details.
- `/gp/product/<ASIN>` product details.
Review data is limited to visible summary ratings/counts and visible histogram rows on search/detail pages. Do not navigate to `/product-reviews`, `/review`, `/gp/customer-reviews`, or review AJAX endpoints.
## Failure Modes
Return a structured warning and do not claim success when any of these happen:
- CAPTCHA or bot-check page.
- Sign-in wall.
- HTTP 429 or 503 that remains after the bounded retry budget.
- Robots rules disallow a planned path.
- Product markup changes enough that required fields cannot be found.
- Amazon returns localized, personalized, or ZIP/session-dependent delivery text that cannot be verified.
## Output Rules
- Unknown fields stay unknown.
- Partial extraction is acceptable only when the response includes warnings and missing-field notes.
- Sponsored products can be returned by default but must be labeled.
- Counts above 30 require operator confirmation or batch splitting.
@@ -0,0 +1,27 @@
# Web-Automation Prompts
Use these patterns when debugging or extending the `amazon-shopping` browser workflow. The TypeScript helper is the default interface; these prompts document the intended rendered-page behavior.
## Search Page
```text
Use the installed web-automation skill. Open https://www.amazon.com/s?k=<encoded query>. Wait for rendered search results. If a CAPTCHA, bot check, sign-in wall, or access denied page appears, stop and return a challenge status. Otherwise extract visible product cards with ASIN, title, product URL, displayed price, rating text, review count text, delivery text, Prime badge/icon/aria/alt signal, sponsor marker, and image URL. Do not open cart, sign-in, wishlist, or review-listing pages.
```
## Detail Page
```text
For each candidate product detail URL under /dp/<ASIN> or /gp/product/<ASIN>, open the page slowly one at a time. Extract title, canonical link, visible buy-box/current price, delivery summary, Prime badge/icon/aria/alt signal, availability, seller when visible, feature bullets, product details/specification table rows, rating score, review count, and visible customer-review histogram percentages if present on the detail page. Do not navigate to /product-reviews or reviewer profile pages.
```
## Pagination
```text
Follow only the visible Amazon pagination control for the next search page, or construct page=<n> only after the current page exposes normal search results and no challenge/block. Stop when enough candidates have been collected, no next page exists, a challenge appears, or maxSearchPages is reached.
```
## Robustness Notes
- Prefer Playwright locator/actionability behavior and bounded waits over fixed sleeps.
- Never follow sponsored redirect URLs, sign-in links, cart links, wishlist links, or review-page links.
- Return partial results with warnings when Amazon markup changes or fields are hidden.
+54
View File
@@ -0,0 +1,54 @@
#!/usr/bin/env node
import { readdir, readFile, stat } from "node:fs/promises";
import { join, relative } from "node:path";
const root = new URL("..", import.meta.url).pathname;
const scannedExtensions = new Set([".md", ".json", ".ts", ".js", ".mjs", ".sh"]);
const installSpecificPath = ["", "Users", "stefano"].join("/");
const forbidden = [
{
pattern: installSpecificPath,
message: "Source files must not hardcode this install path"
}
];
function extensionOf(path) {
const dot = path.lastIndexOf(".");
return dot === -1 ? "" : path.slice(dot);
}
async function walk(dir) {
const entries = await readdir(dir);
const files = [];
for (const entry of entries) {
if (["node_modules", "dist", "coverage", "tmp", "out"].includes(entry)) {
continue;
}
const path = join(dir, entry);
const info = await stat(path);
if (info.isDirectory()) {
files.push(...await walk(path));
} else if (scannedExtensions.has(extensionOf(path)) || entry === "SKILL.md") {
files.push(path);
}
}
return files;
}
const failures = [];
for (const file of await walk(root)) {
const text = await readFile(file, "utf8");
for (const rule of forbidden) {
if (text.includes(rule.pattern)) {
failures.push(`${relative(root, file)}: ${rule.message}`);
}
}
}
if (failures.length > 0) {
for (const failure of failures) {
console.error(failure);
}
process.exitCode = 1;
}
+13
View File
@@ -0,0 +1,13 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SKILL_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
TSX="${SKILL_DIR}/node_modules/.bin/tsx"
if [ ! -x "$TSX" ]; then
echo "Missing local dependencies. Run: cd \"$SKILL_DIR\" && npm install" >&2
exit 127
fi
exec "$TSX" "$SKILL_DIR/src/cli.ts" "$@"
+10
View File
@@ -0,0 +1,10 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SKILL_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
cd "$SKILL_DIR"
npm install
npm run lint
npm test
+228
View File
@@ -0,0 +1,228 @@
import { execFile } from "node:child_process";
import { pathToFileURL } from "node:url";
import { join } from "node:path";
import { promisify } from "node:util";
import { extractDetailPage } from "./detail-page.js";
import { applyFiltersAndLimit } from "./filters.js";
import { createResponse } from "./report.js";
import { extractSearchPage } from "./search-page.js";
import type { ProductSearchResult, SearchProductsRequest, SearchProductsResponse } from "./types.js";
import { resolveWebAutomationRuntime } from "./web-automation-runtime.js";
const execFileAsync = promisify(execFile);
const AMAZON_ROOT = "https://www.amazon.com";
const DEFAULT_WAIT_MS = 4500;
export type HttpClassification = "ok" | "retryable" | "challenge";
interface BrowserDeps {
fetchText?: (url: string) => Promise<string>;
sleep?: (ms: number) => Promise<void>;
now?: () => Date;
}
export function plannedAmazonPaths(asins: string[]): string[] {
return [
"/s",
...asins.flatMap((asin) => [`/dp/${asin}`, `/gp/product/${asin}`])
];
}
export function classifyHttpStatus(status: number | null | undefined): HttpClassification {
if (status === 429 || status === 503) return "retryable";
if (status === 401 || status === 403) return "challenge";
return "ok";
}
export function isPathAllowedByRobots(robots: string, userAgent: string, path: string): boolean {
const groups: Array<{ agents: string[]; disallows: string[] }> = [];
let current: { agents: string[]; disallows: string[] } | undefined;
let hasDirectives = false;
const lines = robots.split(/\r?\n/);
for (const rawLine of lines) {
const line = rawLine.replace(/#.*/, "").trim();
if (!line) continue;
const [rawKey, ...rest] = line.split(":");
const key = rawKey.trim().toLowerCase();
const value = rest.join(":").trim();
if (key === "user-agent") {
if (!current || hasDirectives) {
current = { agents: [], disallows: [] };
groups.push(current);
hasDirectives = false;
}
current.agents.push(value.toLowerCase());
continue;
}
if (key === "disallow") {
hasDirectives = true;
if (current && value) {
current.disallows.push(value);
}
}
}
const normalizedAgent = userAgent.toLowerCase();
const exactGroups = groups.filter((group) => group.agents.includes(normalizedAgent));
const matchedGroups = exactGroups.length > 0 ? exactGroups : groups.filter((group) => group.agents.includes("*"));
const disallows = matchedGroups.flatMap((group) => group.disallows);
return !disallows.some((rule) => path.startsWith(rule));
}
async function defaultFetchText(url: string): Promise<string> {
const response = await fetch(url);
return response.text();
}
async function checkRobots(paths: string[], deps: BrowserDeps): Promise<string[]> {
const warnings: string[] = [];
const robots = await (deps.fetchText ?? defaultFetchText)(`${AMAZON_ROOT}/robots.txt`);
for (const path of paths) {
if (!isPathAllowedByRobots(robots, "*", path)) {
warnings.push(`Amazon robots directives disallow planned path: ${path}`);
}
}
return warnings;
}
async function loadCloakBrowser(runtimeDir: string): Promise<{
ensureBinary?: () => Promise<void>;
launchContext: (options: Record<string, unknown>) => Promise<any>;
}> {
const moduleUrl = pathToFileURL(join(runtimeDir, "node_modules", "cloakbrowser", "dist", "index.js")).toString();
return import(moduleUrl) as Promise<any>;
}
async function checkRuntime(): Promise<string> {
const runtime = await resolveWebAutomationRuntime();
await execFileAsync(runtime.checkInstall.command, runtime.checkInstall.args, { cwd: runtime.checkInstall.cwd });
return runtime.scriptsDir;
}
function searchUrl(query: string, pageNumber: number): string {
const url = new URL("/s", AMAZON_ROOT);
url.searchParams.set("k", query);
if (pageNumber > 1) {
url.searchParams.set("page", String(pageNumber));
}
return url.toString();
}
async function pageHtml(page: any, url: string, deps: BrowserDeps): Promise<{ html: string; status: number | null }> {
let lastStatus: number | null = null;
for (let attempt = 0; attempt < 3; attempt += 1) {
const response = await page.goto(url, { waitUntil: "domcontentloaded", timeout: 45000 });
await page.waitForTimeout?.(DEFAULT_WAIT_MS);
lastStatus = response?.status?.() ?? null;
if (classifyHttpStatus(lastStatus) !== "retryable") {
return {
html: await page.content(),
status: lastStatus
};
}
await (deps.sleep ?? ((ms) => new Promise((resolve) => setTimeout(resolve, ms))))((2 ** attempt) * 1000 + Math.floor(Math.random() * 500));
}
return {
html: await page.content(),
status: lastStatus
};
}
async function enrichDetails(page: any, products: ProductSearchResult[], deps: BrowserDeps): Promise<ProductSearchResult[]> {
const enriched: ProductSearchResult[] = [];
for (const product of products) {
await (deps.sleep ?? ((ms) => new Promise((resolve) => setTimeout(resolve, ms))))(1500 + Math.floor(Math.random() * 1500));
const loaded = await pageHtml(page, product.url, deps);
const classification = classifyHttpStatus(loaded.status);
if (classification === "challenge") {
enriched.push({
...product,
extractionNotes: [...product.extractionNotes, "Detail page returned a challenge/block status."]
});
continue;
}
enriched.push(extractDetailPage(loaded.html, product));
}
return enriched;
}
export async function searchProducts(request: SearchProductsRequest, deps: BrowserDeps = {}): Promise<SearchProductsResponse> {
const warnings: string[] = [];
const robotsWarnings = await checkRobots(plannedAmazonPaths([]), deps);
if (robotsWarnings.length > 0) {
return createResponse({
query: request.query,
filters: request.filters,
limit: request.limit,
maxSearchPages: request.maxSearchPages,
results: [],
filteredOutCount: 0,
warnings: robotsWarnings,
now: deps.now
});
}
const runtimeDir = await checkRuntime();
const cloak = await loadCloakBrowser(runtimeDir);
await cloak.ensureBinary?.();
const context = await cloak.launchContext({
headless: process.env.CLOAKBROWSER_HEADLESS !== "false",
locale: "en-US",
viewport: { width: 1440, height: 900 },
humanize: true
});
const page = await context.newPage();
try {
const candidates: ProductSearchResult[] = [];
let nextUrl: string | undefined = searchUrl(request.query, 1);
for (let pageNumber = 1; pageNumber <= request.maxSearchPages && nextUrl; pageNumber += 1) {
const loaded = await pageHtml(page, nextUrl, deps);
const classification = classifyHttpStatus(loaded.status);
if (classification === "challenge" || classification === "retryable") {
warnings.push(`Amazon returned status ${loaded.status}; stopping without bypass.`);
break;
}
const extracted = extractSearchPage(loaded.html, nextUrl);
warnings.push(...extracted.warnings);
if (extracted.status === "challenge") {
break;
}
candidates.push(...extracted.products);
if (candidates.length >= request.limit * 3) {
break;
}
nextUrl = extracted.nextPageUrl ?? (pageNumber + 1 <= request.maxSearchPages ? searchUrl(request.query, pageNumber + 1) : undefined);
}
let detailCandidates = candidates;
if (!request.skipDetails) {
const detailPaths = plannedAmazonPaths(candidates.map((candidate) => candidate.asin)).filter((path) => path !== "/s");
const detailRobotsWarnings = await checkRobots(detailPaths, deps);
if (detailRobotsWarnings.length > 0) {
warnings.push(...detailRobotsWarnings, "Detail enrichment skipped because robots directives disallow at least one planned detail path.");
} else {
detailCandidates = await enrichDetails(page, candidates.slice(0, request.limit * 3), deps);
}
}
const filtered = applyFiltersAndLimit(detailCandidates, request.filters, request.limit);
return createResponse({
query: request.query,
filters: request.filters,
limit: request.limit,
maxSearchPages: request.maxSearchPages,
results: filtered.results,
filteredOutCount: filtered.filteredOutCount,
warnings,
now: deps.now
});
} finally {
await context.close();
}
}
+211
View File
@@ -0,0 +1,211 @@
#!/usr/bin/env node
import minimist from "minimist";
import { fileURLToPath } from "node:url";
import { searchProducts } from "./browser.js";
import { parseNaturalLanguageRequest } from "./query-parser.js";
import { createMarkdownReport } from "./report.js";
import type { ProductFilters, SearchProductsRequest, SearchProductsResponse } from "./types.js";
export interface CliDeps {
stdout: Pick<NodeJS.WriteStream, "write">;
stderr: Pick<NodeJS.WriteStream, "write">;
now?: () => Date;
searchProducts?: (request: SearchProductsRequest) => Promise<SearchProductsResponse>;
}
export function usage(): string {
return `amazon-shopping
Usage:
scripts/search-products "<product request>" [options]
scripts/search-products --query "<product request>" [options]
Options:
--json Print JSON output
--markdown Print markdown output
--limit N, --max N Maximum products to return (default: 15)
--allow-large-limit Permit limits above 30
--min-rating N Minimum rating score
--min-reviews N Minimum review count
--max-price N Maximum displayed product price
--max-unit-price N Maximum price per unit
--min-width N Minimum product width in inches
--require-prime Require Prime delivery verification
--delivery-by VALUE Require delivery timing, e.g. today, tomorrow, overnight
--max-search-pages N Search result pages to scan, 1-5 (default: 2)
--skip-details Do not open product detail pages
--dry-run Parse and print the planned request without Amazon network access
--help Show this help
`;
}
function parsePositiveInteger(value: unknown, name: string): number | undefined {
if (value === undefined) {
return undefined;
}
const parsed = Number(value);
if (!Number.isInteger(parsed) || parsed < 1) {
throw new Error(`${name} must be an integer greater than 0`);
}
return parsed;
}
function parseNumber(value: unknown, name: string): number | undefined {
if (value === undefined) {
return undefined;
}
const parsed = Number(value);
if (!Number.isFinite(parsed)) {
throw new Error(`${name} must be a number`);
}
return parsed;
}
export function buildSearchUrl(query: string): string {
return `https://www.amazon.com/s?k=${encodeURIComponent(query)}`;
}
export function parseCliRequest(argv: string[]): SearchProductsRequest {
const args = minimist(argv, {
boolean: ["help", "json", "markdown", "allow-large-limit", "dry-run", "skip-details", "require-prime"],
string: [
"query",
"limit",
"max",
"min-rating",
"min-reviews",
"max-price",
"max-unit-price",
"min-width",
"delivery-by",
"sort-by",
"max-search-pages"
],
alias: { h: "help", max: "limit" }
});
const rawQuery = String(args.query ?? args._.join(" ")).trim();
if (!rawQuery) {
throw new Error("A product query is required");
}
const natural = parseNaturalLanguageRequest(rawQuery);
const limit = parsePositiveInteger(args.limit, "limit") ?? natural.limit ?? 15;
if (limit > 30 && !args["allow-large-limit"]) {
throw new Error("Requested limits above 30 require --allow-large-limit or a batched run");
}
const maxSearchPages = parsePositiveInteger(args["max-search-pages"], "max-search-pages") ?? 2;
if (maxSearchPages > 5) {
throw new Error("max-search-pages must be 5 or less");
}
const filters: ProductFilters = { ...natural.filters };
const minRating = parseNumber(args["min-rating"], "min-rating");
const minReviews = parsePositiveInteger(args["min-reviews"], "min-reviews");
const maxPrice = parseNumber(args["max-price"], "max-price");
const maxUnitPrice = parseNumber(args["max-unit-price"], "max-unit-price");
const minWidth = parseNumber(args["min-width"], "min-width");
if (minRating !== undefined) filters.minRating = minRating;
if (minReviews !== undefined) filters.minReviews = minReviews;
if (maxPrice !== undefined) filters.maxPrice = maxPrice;
if (maxUnitPrice !== undefined) filters.maxUnitPrice = maxUnitPrice;
if (minWidth !== undefined) {
filters.minWidthInches = minWidth;
filters.widthComparison = "gte";
}
if (args["require-prime"]) filters.requirePrime = true;
if (args["delivery-by"]) filters.deliveryBy = String(args["delivery-by"]);
if (args["sort-by"]) {
const sortBy = String(args["sort-by"]);
if (sortBy !== "price" && sortBy !== "relevance") {
throw new Error("sort-by must be either price or relevance");
}
filters.sortBy = sortBy;
}
const json = Boolean(args.json);
const markdown = Boolean(args.markdown);
return {
query: natural.query || rawQuery,
filters,
limit,
maxSearchPages,
skipDetails: Boolean(args["skip-details"]),
dryRun: Boolean(args["dry-run"]),
output: json && markdown ? "both" : markdown ? "markdown" : "json"
};
}
function createDryRunResponse(request: SearchProductsRequest, now: () => Date): SearchProductsResponse {
return {
query: request.query,
filters: request.filters,
limit: request.limit,
maxSearchPages: request.maxSearchPages,
results: [],
filteredOutCount: 0,
warnings: [`Dry run only. Planned search URL: ${buildSearchUrl(request.query)}`],
source: {
site: "amazon.com",
scrapedAt: now().toISOString(),
automation: "web-automation/CloakBrowser"
}
};
}
async function defaultSearchProducts(request: SearchProductsRequest, deps: CliDeps): Promise<SearchProductsResponse> {
if (request.dryRun) {
return createDryRunResponse(request, deps.now ?? (() => new Date()));
}
return searchProducts(request, { now: deps.now });
}
function writeResponse(response: SearchProductsResponse, output: SearchProductsRequest["output"], deps: CliDeps): void {
if (output === "markdown") {
deps.stdout.write(createMarkdownReport(response));
return;
}
if (output === "both") {
deps.stdout.write(`${JSON.stringify(response, null, 2)}\n\n${createMarkdownReport(response)}`);
return;
}
deps.stdout.write(`${JSON.stringify(response, null, 2)}\n`);
}
export async function runCli(
argv: string[],
deps: CliDeps = { stdout: process.stdout, stderr: process.stderr }
): Promise<number> {
const rawArgs = minimist(argv, { boolean: ["help"], alias: { h: "help" } });
if (rawArgs.help || argv.length === 0) {
deps.stdout.write(usage());
return 0;
}
try {
const request = parseCliRequest(argv);
const response = deps.searchProducts
? await deps.searchProducts(request)
: await defaultSearchProducts(request, deps);
writeResponse(response, request.output, deps);
return 0;
} catch (error: unknown) {
const message = error instanceof Error ? error.message : String(error);
deps.stderr.write(`${message}\n`);
return 1;
}
}
if (process.argv[1] && fileURLToPath(import.meta.url) === process.argv[1]) {
runCli(process.argv.slice(2)).then((code) => {
process.exitCode = code;
}).catch((error: unknown) => {
const message = error instanceof Error ? error.message : String(error);
process.stderr.write(`${message}\n`);
process.exitCode = 1;
});
}
+160
View File
@@ -0,0 +1,160 @@
import { HTMLElement, parse } from "node-html-parser";
import { parseMoney, parseRating, parseReviewCount, parseStarBreakdown } from "./parsers.js";
import type { DeliverySummary, ProductSearchResult, ProductSpec } from "./types.js";
function textOf(node: HTMLElement | null | undefined): string {
return cleanText(node?.textContent ?? "");
}
function attrOf(node: HTMLElement | null | undefined, name: string): string {
return cleanText(node?.getAttribute(name) ?? "");
}
function cleanText(text: string): string {
return text
.replace(/\s+/g, " ")
.replace(/\s*\{".*$/g, "")
.trim();
}
function isScriptLike(text: string): boolean {
return /\(function\s*\(|window\.|P\.when|ue\.count|tracking\(\)|logShoppableMetrics|buying options|add to cart/i.test(text);
}
function firstText(root: HTMLElement, selectors: string[]): string {
for (const selector of selectors) {
const text = textOf(root.querySelector(selector));
if (text) {
return text;
}
}
return "";
}
function extractBullets(root: HTMLElement): string[] {
const spanBullets = root.querySelectorAll("#feature-bullets li span")
.map((node) => textOf(node))
.filter((text) => text && !/make sure this fits/i.test(text));
if (spanBullets.length > 0) {
return spanBullets;
}
return root.querySelectorAll("#feature-bullets li")
.map((node) => textOf(node))
.filter((text) => text && !/make sure this fits/i.test(text));
}
function extractSpecs(root: HTMLElement): ProductSpec[] {
const specs: ProductSpec[] = [];
const seen = new Set<string>();
const excludedNames = new Set(["customer reviews"]);
for (const row of root.querySelectorAll("tr")) {
const cells = row.querySelectorAll("th,td").map((cell) => textOf(cell)).filter(Boolean);
if (cells.length >= 2) {
const name = cells[0];
const value = cells.slice(1).join(" ");
const key = name.toLowerCase();
if (seen.has(key) || excludedNames.has(key) || isScriptLike(name) || isScriptLike(value)) {
continue;
}
seen.add(key);
specs.push({ name, value });
}
}
return specs;
}
function extractHistogramText(root: HTMLElement): string {
const rows = root.querySelectorAll("#histogramTable tr, [aria-label*='star'] tr");
const parts: string[] = [];
for (const row of rows) {
const cells = row.querySelectorAll("th,td").map((cell) => textOf(cell)).filter(Boolean);
if (cells.length >= 2) {
parts.push(`${cells[0]} ${cells[1]}`);
}
}
return parts.join(" ");
}
function deliveryFromText(text: string, primeSignal = false): DeliverySummary | undefined {
const display = text.replace(/\s+/g, " ").trim();
if (!display) {
return primeSignal ? { display: "Prime delivery available", prime: true } : undefined;
}
return {
display,
free: /\bfree\b/i.test(display),
prime: primeSignal || /\bprime\b/i.test(display)
};
}
function hasPrimeSignal(root: HTMLElement): boolean {
const attributeText = root.querySelectorAll("[id], [class], [aria-label], img[alt]")
.map((node) => [
attrOf(node, "id"),
attrOf(node, "class"),
attrOf(node, "aria-label"),
attrOf(node, "alt")
].join(" "))
.join(" ");
return /a-icon-prime|prime-logo|primeExclusive|primePopover|amazon\s+prime|\bprime\b/i.test(attributeText);
}
function mergeDelivery(detail: DeliverySummary | undefined, base: DeliverySummary | undefined): DeliverySummary | undefined {
if (!detail) {
return base;
}
if (!base) {
return detail;
}
return {
display: detail.display || base.display,
free: Boolean(detail.free || base.free),
prime: Boolean(detail.prime || base.prime),
fastestDate: detail.fastestDate ?? base.fastestDate
};
}
export function extractDetailPage(html: string, base: ProductSearchResult): ProductSearchResult {
const root = parse(html);
const title = firstText(root, ["#productTitle", "h1"]) || base.title;
const priceText = firstText(root, [
"#corePriceDisplay_desktop_feature_div .a-offscreen",
".a-price .a-offscreen",
".a-price"
]);
const deliveryText = firstText(root, [
"#mir-layout-DELIVERY_BLOCK-slot-PRIMARY_DELIVERY_MESSAGE_LARGE",
"#deliveryBlockMessage",
"[data-csa-c-delivery-price]"
]);
const availability = firstText(root, ["#availability", "#availabilityInsideBuyBox_feature_div"]);
const seller = firstText(root, ["#merchant-info", "#sellerProfileTriggerId"]);
const ratingText = attrOf(root.querySelector("#acrPopover"), "title") || textOf(root.querySelector("#acrPopover"));
const reviewText = textOf(root.querySelector("#acrCustomerReviewText"));
const histogram = parseStarBreakdown(extractHistogramText(root));
const product: ProductSearchResult = {
...base,
title,
price: parseMoney(priceText) ?? base.price,
rating: parseRating(ratingText) ?? base.rating,
reviewCount: parseReviewCount(reviewText) ?? base.reviewCount,
delivery: mergeDelivery(deliveryFromText(deliveryText, hasPrimeSignal(root)), base.delivery),
availability: availability || base.availability,
seller: seller || base.seller,
bullets: extractBullets(root),
specs: extractSpecs(root),
starBreakdown: histogram ?? base.starBreakdown,
missingFields: [...base.missingFields],
extractionNotes: [...base.extractionNotes]
};
for (const field of ["price", "delivery", "rating", "reviewCount", "starBreakdown"] as const) {
if (product[field] === undefined && !product.missingFields.includes(field)) {
product.missingFields.push(field);
}
}
return product;
}
+123
View File
@@ -0,0 +1,123 @@
import type { FilteredProducts, ProductFilters, ProductSearchResult } from "./types.js";
import { extractWidthInches } from "./product-metrics.js";
function passesMin(value: number | undefined, threshold: number, comparison: "gt" | "gte" | undefined): boolean {
if (value === undefined) {
return false;
}
return comparison === "gt" ? value > threshold : value >= threshold;
}
function filterReasons(product: ProductSearchResult, filters: ProductFilters): string[] {
const reasons: string[] = [];
if (filters.minRating !== undefined && !passesMin(product.rating, filters.minRating, filters.ratingComparison)) {
reasons.push(product.rating === undefined ? "rating unknown" : `rating ${product.rating} below filter`);
}
if (filters.minReviews !== undefined && !passesMin(product.reviewCount, filters.minReviews, filters.reviewCountComparison)) {
reasons.push(product.reviewCount === undefined ? "review count unknown" : `review count ${product.reviewCount} below filter`);
}
if (filters.maxPrice !== undefined) {
if (!product.price) {
reasons.push("price unknown");
} else if (product.price.amount > filters.maxPrice) {
reasons.push(`price ${product.price.display} above filter`);
}
}
if (filters.maxUnitPrice !== undefined) {
if (!product.unitPrice) {
reasons.push("unit price unknown");
} else if (product.unitPrice.amount > filters.maxUnitPrice) {
reasons.push(`unit price ${product.unitPrice.display} above filter`);
}
}
if (filters.minWidthInches !== undefined) {
const width = extractWidthInches(product);
if (width === undefined) {
reasons.push("width unknown");
} else if (!passesMin(width, filters.minWidthInches, filters.widthComparison)) {
reasons.push(`width ${width} inches below filter`);
}
}
if (filters.requirePrime && !product.delivery?.prime) {
reasons.push("Prime delivery not verified");
}
if (filters.requireFreeDelivery && !product.delivery?.free) {
reasons.push("free delivery not verified");
}
if (filters.deliveryBy && !deliveryMatches(product.delivery?.display, filters.deliveryBy)) {
reasons.push(`${filters.deliveryBy} delivery not verified`);
}
return reasons;
}
function deliveryMatches(display: string | undefined, deliveryBy: string): boolean {
if (!display) {
return false;
}
const normalized = display.toLowerCase();
if (deliveryBy === "today") {
return /\btoday\b|same[- ]day/.test(normalized);
}
if (deliveryBy === "tomorrow" || deliveryBy === "overnight") {
return /\btomorrow\b|overnight|next[- ]day|one[- ]day/.test(normalized);
}
return normalized.includes(deliveryBy.toLowerCase());
}
function comparisonSymbol(comparison: "gt" | "gte" | undefined): string {
return comparison === "gt" ? ">" : ">=";
}
function rankProducts(a: ProductSearchResult, b: ProductSearchResult, filters: ProductFilters): number {
if (filters.sortBy === "price") {
return (a.price?.amount ?? Number.POSITIVE_INFINITY) - (b.price?.amount ?? Number.POSITIVE_INFINITY);
}
const ratingDiff = (b.rating ?? -1) - (a.rating ?? -1);
if (ratingDiff !== 0) return ratingDiff;
const reviewDiff = (b.reviewCount ?? -1) - (a.reviewCount ?? -1);
if (reviewDiff !== 0) return reviewDiff;
return (a.price?.amount ?? Number.POSITIVE_INFINITY) - (b.price?.amount ?? Number.POSITIVE_INFINITY);
}
export function applyFiltersAndLimit(
products: ProductSearchResult[],
filters: ProductFilters,
limit: number
): FilteredProducts {
const filteredOutReasons: Record<string, string[]> = {};
const uniqueProducts = new Map<string, ProductSearchResult>();
for (const product of products) {
if (!uniqueProducts.has(product.asin)) {
uniqueProducts.set(product.asin, product);
}
}
const passing: ProductSearchResult[] = [];
for (const product of uniqueProducts.values()) {
const reasons = filterReasons(product, filters);
if (reasons.length > 0) {
filteredOutReasons[product.asin] = reasons;
continue;
}
passing.push({
...product,
matchedFilters: [
...product.matchedFilters,
...(filters.minRating !== undefined ? [`rating ${filters.ratingComparison ?? "gte"} ${filters.minRating}`] : []),
...(filters.minReviews !== undefined ? [`reviews ${filters.reviewCountComparison ?? "gte"} ${filters.minReviews}`] : []),
...(filters.maxPrice !== undefined ? [`price <= ${filters.maxPrice}`] : []),
...(filters.maxUnitPrice !== undefined ? [`unit price <= ${filters.maxUnitPrice}`] : []),
...(filters.minWidthInches !== undefined ? [`width ${comparisonSymbol(filters.widthComparison)} ${filters.minWidthInches} inches`] : []),
...(filters.requirePrime ? ["Prime delivery"] : []),
...(filters.requireFreeDelivery ? ["free delivery"] : []),
...(filters.deliveryBy ? [`delivery by ${filters.deliveryBy}`] : [])
]
});
}
return {
results: passing.sort((a, b) => rankProducts(a, b, filters)).slice(0, limit),
filteredOutCount: uniqueProducts.size - passing.length,
filteredOutReasons
};
}
+118
View File
@@ -0,0 +1,118 @@
import type { MoneyValue, StarBreakdown, UnitCountExtraction } from "./types.js";
export function parseMoney(text: string | undefined | null): MoneyValue | undefined {
if (!text) {
return undefined;
}
const compact = text.replace(/\s+/g, " ").trim();
const match = compact.match(/\$\s*([0-9][0-9,]*(?:\.[0-9]{1,2})?)/);
if (!match) {
return undefined;
}
const amount = Number(match[1].replace(/,/g, ""));
if (!Number.isFinite(amount)) {
return undefined;
}
return {
amount,
currency: "USD",
display: compact
};
}
export function parseUnitPrice(text: string | undefined | null): MoneyValue | undefined {
if (!text) {
return undefined;
}
const compact = text.replace(/\s+/g, " ").trim();
const unitMatch = compact.match(/(\$\s*[0-9][0-9,]*(?:\.[0-9]{1,2})?)(?:\s*\$\s*[0-9][0-9,]*(?:\.[0-9]{1,2})?)?\s*(?:\/|\bper\b\s*)\s*(?:count|unit|item|piece|pack|each)\b/i)
?? compact.match(/(\$\s*[0-9][0-9,]*(?:\.[0-9]{1,2})?)\s*(?:each)\b/i);
if (!unitMatch) {
return undefined;
}
const display = unitMatch[0]
.replace(/\$\s*([0-9][0-9,]*(?:\.[0-9]{1,2})?)\s*\$\s*\1/i, "$$$1")
.replace(/\s+/g, "");
return parseMoney(display);
}
export function parseRating(text: string | undefined | null): number | undefined {
if (!text) {
return undefined;
}
const match = text.match(/([0-5](?:\.[0-9])?)\s*(?:out of\s*)?5\s*stars?/i)
?? text.match(/\brated\s+([0-5](?:\.[0-9])?)/i);
if (!match) {
return undefined;
}
const rating = Number(match[1]);
return Number.isFinite(rating) ? rating : undefined;
}
export function parseReviewCount(text: string | undefined | null): number | undefined {
if (!text) {
return undefined;
}
const match = text.match(/([0-9][0-9,]*)\s*(?:ratings?|reviews?)/i);
if (!match) {
return undefined;
}
const count = Number(match[1].replace(/,/g, ""));
return Number.isInteger(count) ? count : undefined;
}
export function parseStarBreakdown(text: string | undefined | null): StarBreakdown | undefined {
if (!text) {
return undefined;
}
const breakdown: Partial<Omit<StarBreakdown, "basis">> = {};
const words: Record<string, keyof Omit<StarBreakdown, "basis">> = {
"5": "five",
"4": "four",
"3": "three",
"2": "two",
"1": "one"
};
const percentMatches = [...text.matchAll(/([1-5])\s*star\s*([0-9]{1,3})\s*%/gi)];
if (percentMatches.length === 0) {
return undefined;
}
for (const match of percentMatches) {
const key = words[match[1]];
if (key) {
breakdown[key] = Number(match[2]);
}
}
return {
...breakdown,
basis: "percent"
};
}
export function extractUnitCount(text: string | undefined | null): UnitCountExtraction | undefined {
if (!text) {
return undefined;
}
const patterns = [
{ pattern: /(\d{1,4})\s*[- ]?(?:count|ct)\b/i, confidence: "high" as const },
{ pattern: /\bpack\s+of\s+(\d{1,4})\b/i, confidence: "high" as const },
{ pattern: /\b(\d{1,4})\s*[- ]?pack\b/i, confidence: "high" as const },
{ pattern: /\bset\s+of\s+(\d{1,4})\b/i, confidence: "medium" as const },
{ pattern: /\b(\d{1,4})\s+(?:bulbs?|cables?|pieces?|pcs)\b/i, confidence: "low" as const }
];
for (const { pattern, confidence } of patterns) {
const match = text.match(pattern);
if (!match) {
continue;
}
const count = Number(match[1]);
if (Number.isInteger(count) && count > 0) {
return {
count,
confidence,
source: match[0]
};
}
}
return undefined;
}
@@ -0,0 +1,62 @@
import type { ProductSearchResult, ProductSpec } from "./types.js";
function parseDimensionNumber(text: string): number | undefined {
const match = text.match(/([0-9]+(?:\.[0-9]+)?)/);
return match ? Number(match[1]) : undefined;
}
function isOverallWidthSpec(spec: ProductSpec): boolean {
const name = spec.name.toLowerCase();
if (/seat|arm|door|package|box|back|cushion/.test(name)) {
return false;
}
return /width|dimensions?/.test(name);
}
function widthFromSpec(spec: ProductSpec): number | undefined {
if (!isOverallWidthSpec(spec)) {
return undefined;
}
const name = spec.name.toLowerCase();
const value = spec.value;
const labeledWidth = value.match(/([0-9]+(?:\.[0-9]+)?)\s*(?:"|in(?:ches?)?)?\s*W\b/i);
if (labeledWidth) {
return Number(labeledWidth[1]);
}
if (/width/.test(name)) {
return parseDimensionNumber(value);
}
const orderMatch = name.match(/\b([dwh])\s*x\s*([dwh])(?:\s*x\s*([dwh]))?\b/i);
if (orderMatch) {
const order = orderMatch.slice(1).filter(Boolean).map((part) => part.toLowerCase());
const widthIndex = order.indexOf("w");
const values = value.match(/[0-9]+(?:\.[0-9]+)?/g)?.map(Number) ?? [];
if (widthIndex >= 0 && values[widthIndex] !== undefined) {
return values[widthIndex];
}
}
return undefined;
}
export function extractWidthInches(product: ProductSearchResult): number | undefined {
for (const spec of product.specs) {
const width = widthFromSpec(spec);
if (width !== undefined) {
return width;
}
}
const titleMatch = product.title.match(/\b([0-9]+(?:\.[0-9]+)?)\s*(?:["”]|in(?:ch(?:es)?)?)\b/i);
return titleMatch ? Number(titleMatch[1]) : undefined;
}
export function formatWidthInches(width: number | undefined): string {
if (width === undefined) {
return "unknown";
}
return `${Number.isInteger(width) ? width.toFixed(0) : width.toFixed(1)}"`;
}
+111
View File
@@ -0,0 +1,111 @@
import type { ParsedNaturalLanguageRequest, ProductFilters } from "./types.js";
function cleanQuery(text: string): string {
return text
.replace(/\breview score of\b/gi, " ")
.replace(/\brating of\b/gi, " ")
.replace(/\b(?:delivery|shipping)\s+only\b/gi, " ")
.replace(/\blow\s+to\s+high\b/gi, " ")
.replace(/\bhigh\s+to\s+low\b/gi, " ")
.replace(/\bof\s+in\s+width\b/gi, " ")
.replace(/\bin\s+width\b/gi, " ")
.replace(/\b(?:that|and|with|have)\b/gi, " ")
.replace(/[,\s]+/g, " ")
.replace(/\s+/g, " ")
.replace(/\s+(and|or|a)$/i, "")
.trim();
}
function removeMatched(text: string, match: RegExpMatchArray | null): string {
if (!match) {
return text;
}
return text.replace(match[0], " ");
}
export function parseNaturalLanguageRequest(input: string): ParsedNaturalLanguageRequest {
let remaining = input.trim();
const filters: ProductFilters = {
includeKeywords: [],
excludeKeywords: []
};
let limit: number | undefined;
const limitMatch = remaining.match(/\b(?:return|limit|top)\s+(\d{1,3})\b/i);
if (limitMatch) {
limit = Number(limitMatch[1]);
remaining = removeMatched(remaining, limitMatch);
}
const sortByPriceMatch = remaining.match(/\b(?:by price|sort(?:ed)? by price|lowest price|cheapest|least expensive)\b/i);
if (sortByPriceMatch) {
filters.sortBy = "price";
remaining = removeMatched(remaining, sortByPriceMatch);
}
const deliveryTomorrowMatch = remaining.match(/\b(?:delivery|delivered|arrives?|shipping|ships?)\s+(?:by\s+)?tomorrow\b/i);
const deliveryTodayMatch = remaining.match(/\b(?:delivery|delivered|arrives?|shipping|ships?)\s+(?:by\s+)?today\b/i)
?? remaining.match(/\bsame[- ]day\s+(?:delivery|shipping)\b/i);
const overnightMatch = remaining.match(/\bovernight\s+(?:delivery|shipping)\b/i)
?? remaining.match(/\bnext[- ]day\s+(?:delivery|shipping)\b/i);
const deliveryMatch = overnightMatch ?? deliveryTomorrowMatch ?? deliveryTodayMatch;
if (deliveryMatch) {
filters.deliveryBy = overnightMatch ? "overnight" : deliveryTomorrowMatch ? "tomorrow" : "today";
remaining = removeMatched(remaining, deliveryMatch);
}
const primeMatch = remaining.match(/\b(?:(?:shipped|ships|shipping|delivery|delivered)\s+(?:with|by|from)\s+)?prime\b/i);
if (primeMatch) {
filters.requirePrime = true;
remaining = removeMatched(remaining, primeMatch);
}
const widthMatch = remaining.match(/\b(?:width\s*(?:of\s*)?)?([0-9]+(?:\.[0-9]+)?)\s*(?:inches|inch|in\.?|")\s*(?:or\s+)?(?:wider|wide|larger|longer)\b/i)
?? remaining.match(/\b([0-9]+(?:\.[0-9]+)?)\s*(?:inches|inch|in\.?|")\s*(?:or\s+)?(?:wider|wide|larger|longer)\s+(?:in\s+)?width\b/i)
?? remaining.match(/\b(?:at least|minimum|min\.?)\s+([0-9]+(?:\.[0-9]+)?)\s*(?:inches|inch|in\.?|")\s+(?:wide|width)\b/i);
if (widthMatch) {
filters.minWidthInches = Number(widthMatch[1]);
filters.widthComparison = "gte";
remaining = removeMatched(remaining, widthMatch);
}
const unitPriceMatch = remaining.match(/\b(?:cost\s+)?(?:less than|under|below)\s+\$([0-9]+(?:\.[0-9]{1,2})?)\s*(?:each|per\b|\/\s*(?:count|unit|item))\b/i);
if (unitPriceMatch) {
filters.maxUnitPrice = Number(unitPriceMatch[1]);
remaining = removeMatched(remaining, unitPriceMatch);
}
const maxPriceMatch = remaining.match(/\b(?:cost\s+)?(?:less than|under|below)\s+\$([0-9]+(?:\.[0-9]{1,2})?)\b/i);
if (maxPriceMatch) {
filters.maxPrice = Number(maxPriceMatch[1]);
remaining = removeMatched(remaining, maxPriceMatch);
}
const exclusiveReviews = remaining.match(/\b(?:over|more than|above)\s+([0-9][0-9,]*)\s*(?:reviews?|ratings?)\b/i);
const inclusiveReviews = remaining.match(/\b(?:at least|minimum|min\.?)\s+([0-9][0-9,]*)\s*(?:reviews?|ratings?)\b/i)
?? remaining.match(/\b([0-9][0-9,]*)\s*\+\s*(?:reviews?|ratings?)\b/i);
const reviewMatch = exclusiveReviews ?? inclusiveReviews;
if (reviewMatch) {
filters.minReviews = Number(reviewMatch[1].replace(/,/g, ""));
filters.reviewCountComparison = exclusiveReviews ? "gt" : "gte";
remaining = removeMatched(remaining, reviewMatch);
}
const exclusiveRating = remaining.match(/\b(?:a\s+)?(?:(?:review score|rating)\s+of\s+|rating\s+)?(?:more than|over|above|rated above)\s+([0-5](?:\.[0-9])?)\s*(?:stars?)?\b/i);
const inclusiveRating = remaining.match(/\b(?:a\s+)?(?:review score|rating)(?:\s+of)?\s+([0-5](?:\.[0-9])?)\s*(?:stars?)?\s+(?:or|and)\s+(?:higher|better)\b/i)
?? remaining.match(/\b([0-5](?:\.[0-9])?)\s*stars?\s+or\s+better\b/i)
?? remaining.match(/\b([0-5](?:\.[0-9])?)\s*stars?\s+(?:or|and)\s+(?:higher|better)\b/i)
?? remaining.match(/\b(?:at least|minimum|min\.?)\s+([0-5](?:\.[0-9])?)\s*(?:stars?|rating)?\b/i);
const ratingMatch = exclusiveRating ?? inclusiveRating;
if (ratingMatch) {
filters.minRating = Number(ratingMatch[1]);
filters.ratingComparison = exclusiveRating ? "gt" : "gte";
remaining = removeMatched(remaining, ratingMatch);
}
return {
query: cleanQuery(remaining),
filters,
limit
};
}
+131
View File
@@ -0,0 +1,131 @@
import type { ProductFilters, ProductSearchResult, SearchProductsResponse } from "./types.js";
import { extractWidthInches, formatWidthInches } from "./product-metrics.js";
export interface ResponseInput {
query: string;
filters: ProductFilters;
limit: number;
maxSearchPages: number;
results: ProductSearchResult[];
filteredOutCount: number;
warnings: string[];
now?: () => Date;
}
export function createResponse(input: ResponseInput): SearchProductsResponse {
return {
query: input.query,
filters: input.filters,
limit: input.limit,
maxSearchPages: input.maxSearchPages,
results: input.results,
filteredOutCount: input.filteredOutCount,
warnings: input.warnings,
source: {
site: "amazon.com",
scrapedAt: (input.now ?? (() => new Date()))().toISOString(),
automation: "web-automation/CloakBrowser"
}
};
}
function formatFilters(filters: ProductFilters): string {
const parts = [
filters.minRating !== undefined ? `rating ${filters.ratingComparison ?? "gte"} ${filters.minRating}` : "",
filters.minReviews !== undefined ? `reviews ${filters.reviewCountComparison ?? "gte"} ${filters.minReviews}` : "",
filters.maxPrice !== undefined ? `price <= $${filters.maxPrice}` : "",
filters.maxUnitPrice !== undefined ? `unit price <= $${filters.maxUnitPrice}` : "",
filters.minWidthInches !== undefined ? `width ${filters.widthComparison ?? "gte"} ${filters.minWidthInches} inches` : "",
filters.requirePrime ? "Prime delivery" : "",
filters.requireFreeDelivery ? "free delivery" : "",
filters.deliveryBy ? `delivery by ${filters.deliveryBy}` : "",
filters.sortBy === "price" ? "sort by price" : ""
].filter(Boolean);
return parts.length > 0 ? parts.join(", ") : "none";
}
function compactText(value: string): string {
return value.replace(/\s+/g, " ").trim();
}
function marker(passes: boolean | undefined, enabled: boolean): string {
if (!enabled) {
return "";
}
return passes ? " OK" : " NO";
}
function widthCell(product: ProductSearchResult, filters: ProductFilters): string {
const width = extractWidthInches(product);
const passes = width !== undefined && (filters.widthComparison === "gt" ? width > (filters.minWidthInches ?? 0) : width >= (filters.minWidthInches ?? 0));
return `${formatWidthInches(width)}${marker(passes, filters.minWidthInches !== undefined)}`;
}
function primeCell(product: ProductSearchResult, filters: ProductFilters): string {
if (product.delivery?.prime) {
return `Prime${marker(true, Boolean(filters.requirePrime))}`;
}
return `not verified${marker(false, Boolean(filters.requirePrime))}`;
}
function deliveryCell(product: ProductSearchResult, filters: ProductFilters): string {
const display = product.delivery?.display ?? "unknown";
if (!filters.deliveryBy) {
return display;
}
const normalized = display.toLowerCase();
const passes = filters.deliveryBy === "today"
? /\btoday\b|same[- ]day/.test(normalized)
: filters.deliveryBy === "tomorrow" || filters.deliveryBy === "overnight"
? /\btomorrow\b|overnight|next[- ]day|one[- ]day/.test(normalized)
: normalized.includes(filters.deliveryBy.toLowerCase());
return `${display}${marker(passes, true)}`;
}
function resultBlocks(products: ProductSearchResult[], filters: ProductFilters): string[] {
return products.flatMap((product, index) => [
`${index + 1}. ${compactText(product.title)}`,
`Price: ${product.price?.display ?? "unknown"}`,
`Rating: ${product.rating ?? "unknown"} stars`,
`Reviews: ${product.reviewCount?.toLocaleString("en-US") ?? "unknown"}`,
`Width: ${widthCell(product, filters)}`,
`Prime: ${primeCell(product, filters)}`,
`Delivery: ${compactText(deliveryCell(product, filters))}`,
`Link: ${product.url}`,
""
]);
}
function metadataLines(products: ProductSearchResult[]): string[] {
const lines: string[] = [];
for (const product of products) {
const notes = [
product.missingFields.length > 0 ? `missing ${product.missingFields.join(", ")}` : "",
product.isSponsored ? "sponsored" : "",
product.extractionNotes.length > 0 ? product.extractionNotes.join("; ") : ""
].filter(Boolean);
if (notes.length > 0) {
lines.push(`- ${product.title}: ${notes.join("; ")}`);
}
}
return lines;
}
export function createMarkdownReport(response: SearchProductsResponse): string {
const lines = [
`# Amazon Shopping Results`,
"",
`Query: ${response.query}`,
`Filters: ${formatFilters(response.filters)}`,
`Results returned: ${response.results.length} (filtered out: ${response.filteredOutCount})`,
response.warnings.length > 0 ? `Warnings: ${response.warnings.join("; ")}` : "",
"",
"## Best Matches",
"",
response.results.length > 0 ? "" : "No products matched all requested filters.",
...resultBlocks(response.results, response.filters),
"",
...metadataLines(response.results)
].filter((line) => line !== "");
return `${lines.join("\n")}\n`;
}
+155
View File
@@ -0,0 +1,155 @@
import { HTMLElement, parse } from "node-html-parser";
import { parseMoney, parseRating, parseReviewCount, parseUnitPrice } from "./parsers.js";
import type { DeliverySummary, ProductSearchResult, SearchPageExtraction } from "./types.js";
function textOf(node: HTMLElement | null | undefined): string {
return node?.textContent.replace(/\s+/g, " ").trim() ?? "";
}
function attrOf(node: HTMLElement | null | undefined, name: string): string | undefined {
return node?.getAttribute(name) ?? undefined;
}
function absoluteAmazonUrl(href: string | undefined, currentUrl = "https://www.amazon.com/"): string | undefined {
if (!href) {
return undefined;
}
if (href.startsWith("https://www.amazon.com")) {
return href;
}
try {
const parsed = new URL(href, currentUrl);
if (parsed.hostname !== "www.amazon.com") {
return undefined;
}
return parsed.toString();
} catch {
return undefined;
}
}
function normalizeProductUrl(asin: string, href: string | undefined, currentUrl: string): string {
const absolute = absoluteAmazonUrl(href, currentUrl);
if (!absolute) {
return `https://www.amazon.com/dp/${asin}`;
}
try {
const url = new URL(absolute);
const match = url.pathname.match(/\/(?:dp|gp\/product)\/([A-Z0-9]{8,14})/i);
if (match) {
return `https://www.amazon.com/dp/${match[1].toUpperCase()}`;
}
} catch {
return `https://www.amazon.com/dp/${asin}`;
}
return `https://www.amazon.com/dp/${asin}`;
}
function detectChallenge(html: string): boolean {
return /robot check|enter the characters you see|captcha|automated access|access denied/i.test(html);
}
function deliveryFromText(text: string, primeSignal = false): DeliverySummary | undefined {
const compact = text.replace(/\s+/g, " ").trim();
const deliveryMatch = compact.match(/((?:FREE\s+)?delivery\b[^.]+)/i);
if (!deliveryMatch) {
return primeSignal ? { display: "Prime delivery available", prime: true } : undefined;
}
const display = deliveryMatch[1].trim();
return {
display,
free: /\bfree\b/i.test(display),
prime: primeSignal || /\bprime\b/i.test(display)
};
}
function hasPrimeSignal(card: HTMLElement): boolean {
const attributeText = card.querySelectorAll("[class], [aria-label], img[alt]")
.map((node) => [
attrOf(node, "class") ?? "",
attrOf(node, "aria-label") ?? "",
attrOf(node, "alt") ?? ""
].join(" "))
.join(" ");
return /a-icon-prime|prime-logo|amazon\s+prime|\bprime\b/i.test(attributeText);
}
function firstText(card: HTMLElement, selectors: string[]): string {
for (const selector of selectors) {
const value = textOf(card.querySelector(selector));
if (value) {
return value;
}
}
return "";
}
function firstUnitPriceText(card: HTMLElement): string {
for (const node of card.querySelectorAll(".a-color-secondary, .a-size-base, span")) {
const value = textOf(node);
if (parseUnitPrice(value)) {
return value;
}
}
return "";
}
export function extractSearchPage(html: string, currentUrl: string): SearchPageExtraction {
if (detectChallenge(html)) {
return {
status: "challenge",
products: [],
warnings: ["Amazon returned a challenge or blocked page; stopping without bypass."],
};
}
const root = parse(html);
const cards = root.querySelectorAll("[data-asin]")
.filter((card) => /^[A-Z0-9]{8,14}$/i.test(card.getAttribute("data-asin") ?? ""));
const products: ProductSearchResult[] = [];
for (const card of cards) {
const asin = (card.getAttribute("data-asin") ?? "").toUpperCase();
const link = card.querySelector("h2 a") ?? card.querySelector("a[href*='/dp/']") ?? card.querySelector("a[href*='/gp/product/']");
const title = textOf(link) || firstText(card, ["h2", "[data-cy='title-recipe']"]);
if (!title) {
continue;
}
const priceText = firstText(card, [".a-price .a-offscreen", ".a-price"]);
const allText = textOf(card);
const unitPriceText = firstUnitPriceText(card);
const ariaText = card.querySelectorAll("[aria-label]")
.map((node) => attrOf(node, "aria-label") ?? "")
.join(" ");
const delivery = deliveryFromText(allText, hasPrimeSignal(card));
const product: ProductSearchResult = {
asin,
title,
url: normalizeProductUrl(asin, attrOf(link, "href"), currentUrl),
imageUrl: attrOf(card.querySelector("img"), "src"),
price: parseMoney(priceText),
unitPrice: parseUnitPrice(unitPriceText),
rating: parseRating(ariaText || allText),
reviewCount: parseReviewCount(ariaText || allText),
delivery,
specs: [],
bullets: [],
isSponsored: /\bsponsored\b/i.test(allText),
matchedFilters: [],
missingFields: [],
extractionNotes: []
};
products.push(product);
}
const nextHref = attrOf(root.querySelector(".s-pagination-next[href]"), "href");
const nextPageUrl = absoluteAmazonUrl(nextHref, currentUrl);
return {
status: "ok",
products,
warnings: [],
nextPageUrl: nextPageUrl ?? undefined
};
}
+114
View File
@@ -0,0 +1,114 @@
export interface SearchProductsRequest {
query: string;
filters: ProductFilters;
limit: number;
maxSearchPages: number;
skipDetails: boolean;
dryRun: boolean;
output: "json" | "markdown" | "both";
}
export interface ProductFilters {
minRating?: number;
ratingComparison?: "gt" | "gte";
minReviews?: number;
reviewCountComparison?: "gt" | "gte";
maxPrice?: number;
maxUnitPrice?: number;
minWidthInches?: number;
widthComparison?: "gt" | "gte";
includeKeywords: string[];
excludeKeywords: string[];
requirePrime?: boolean;
requireFreeDelivery?: boolean;
deliveryBy?: string;
sortBy?: "relevance" | "price";
}
export interface ProductSearchResult {
asin: string;
title: string;
url: string;
imageUrl?: string;
price?: MoneyValue;
unitPrice?: MoneyValue;
rating?: number;
reviewCount?: number;
starBreakdown?: StarBreakdown;
delivery?: DeliverySummary;
specs: ProductSpec[];
bullets: string[];
seller?: string;
isSponsored?: boolean;
availability?: string;
matchedFilters: string[];
missingFields: string[];
extractionNotes: string[];
}
export interface MoneyValue {
amount: number;
currency: "USD";
display: string;
}
export interface DeliverySummary {
display: string;
prime?: boolean;
free?: boolean;
fastestDate?: string;
}
export interface StarBreakdown {
five?: number;
four?: number;
three?: number;
two?: number;
one?: number;
basis: "percent" | "count";
}
export interface ProductSpec {
name: string;
value: string;
}
export interface SearchProductsResponse {
query: string;
filters: ProductFilters;
limit: number;
maxSearchPages: number;
results: ProductSearchResult[];
filteredOutCount: number;
warnings: string[];
source: {
site: "amazon.com";
scrapedAt: string;
automation: "web-automation/CloakBrowser";
};
}
export interface ParsedNaturalLanguageRequest {
query: string;
filters: ProductFilters;
limit?: number;
}
export interface UnitCountExtraction {
count: number;
confidence: "high" | "medium" | "low";
source: string;
}
export interface SearchPageExtraction {
status: "ok" | "challenge";
products: ProductSearchResult[];
warnings: string[];
nextPageUrl?: string;
}
export interface FilteredProducts {
results: ProductSearchResult[];
filteredOutCount: number;
filteredOutReasons: Record<string, string[]>;
}
@@ -0,0 +1,71 @@
import { access } from "node:fs/promises";
import { constants } from "node:fs";
import { dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
export interface RuntimeResolverOptions {
env?: NodeJS.ProcessEnv;
homeDir?: string;
skillDir?: string;
}
export interface WebAutomationRuntime {
scriptsDir: string;
checkInstall: {
cwd: string;
command: string;
args: string[];
};
}
async function assertFile(path: string, label: string): Promise<void> {
try {
await access(path, constants.F_OK);
} catch {
throw new Error(`web-automation runtime is missing ${label}: ${path}`);
}
}
async function assertExecutableOrFile(path: string, label: string): Promise<void> {
try {
await access(path, constants.X_OK);
} catch {
await assertFile(path, label);
}
}
function defaultSkillDir(): string {
return resolve(dirname(fileURLToPath(import.meta.url)), "..");
}
export async function resolveWebAutomationRuntime(options: RuntimeResolverOptions = {}): Promise<WebAutomationRuntime> {
const env = options.env ?? process.env;
const homeDir = options.homeDir ?? process.env.HOME ?? "";
const skillDir = options.skillDir ?? defaultSkillDir();
const candidates = [
env.AMAZON_SHOPPING_WEB_AUTOMATION_DIR,
homeDir ? join(homeDir, ".openclaw", "workspace", "skills", "web-automation", "scripts") : undefined,
resolve(skillDir, "..", "web-automation", "scripts")
].filter((candidate): candidate is string => Boolean(candidate));
const errors: string[] = [];
for (const scriptsDir of candidates) {
try {
await assertFile(join(scriptsDir, "check-install.js"), "check-install.js");
await assertFile(join(scriptsDir, "package.json"), "package.json");
await assertExecutableOrFile(join(scriptsDir, "node_modules", ".bin", "tsx"), "node_modules/.bin/tsx");
return {
scriptsDir,
checkInstall: {
cwd: scriptsDir,
command: "node",
args: ["check-install.js"]
}
};
} catch (error: unknown) {
errors.push(error instanceof Error ? error.message : String(error));
}
}
throw new Error(`Unable to locate usable web-automation runtime.\n${errors.join("\n")}`);
}
@@ -0,0 +1,43 @@
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import { classifyHttpStatus, isPathAllowedByRobots, plannedAmazonPaths } from "../src/browser.js";
describe("browser compliance helpers", () => {
it("plans only search and product-detail paths", () => {
assert.deepEqual(plannedAmazonPaths(["B0TEST0001"]), ["/s", "/dp/B0TEST0001", "/gp/product/B0TEST0001"]);
});
it("honors robots disallow rules for planned paths", () => {
const robots = `
User-agent: *
Disallow: /cart
Disallow: /product-reviews
Disallow: /dp/private
`;
assert.equal(isPathAllowedByRobots(robots, "*", "/s"), true);
assert.equal(isPathAllowedByRobots(robots, "*", "/product-reviews/B0TEST0001"), false);
assert.equal(isPathAllowedByRobots(robots, "*", "/dp/private/B0TEST0001"), false);
});
it("does not leak disallow rules from other user-agent groups", () => {
const robots = `
User-agent: specialbot
Disallow: /dp
User-agent: *
Disallow: /cart
`;
assert.equal(isPathAllowedByRobots(robots, "*", "/dp/B0TEST0001"), true);
assert.equal(isPathAllowedByRobots(robots, "specialbot", "/dp/B0TEST0001"), false);
});
it("classifies retryable and challenge statuses", () => {
assert.equal(classifyHttpStatus(429), "retryable");
assert.equal(classifyHttpStatus(503), "retryable");
assert.equal(classifyHttpStatus(403), "challenge");
assert.equal(classifyHttpStatus(200), "ok");
});
});
+132
View File
@@ -0,0 +1,132 @@
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import { buildSearchUrl, parseCliRequest, runCli } from "../src/cli.js";
function createOutput() {
let stdout = "";
let stderr = "";
return {
stdout: { write: (chunk: string) => { stdout += chunk; return true; } },
stderr: { write: (chunk: string) => { stderr += chunk; return true; } },
get stdoutText() { return stdout; },
get stderrText() { return stderr; }
};
}
describe("amazon-shopping CLI", () => {
it("prints help", async () => {
const output = createOutput();
const code = await runCli(["--help"], output);
assert.equal(code, 0);
assert.match(output.stdoutText, /scripts\/search-products/);
assert.match(output.stdoutText, /--dry-run/);
});
it("defaults to 15 results and two search pages", () => {
const request = parseCliRequest(["usb c cable"]);
assert.equal(request.query, "usb c cable");
assert.equal(request.limit, 15);
assert.equal(request.maxSearchPages, 2);
assert.equal(request.output, "json");
});
it("maps kebab-case CLI filters into the request contract", () => {
const request = parseCliRequest([
"--query",
"100w led bulbs",
"--min-rating",
"4.5",
"--min-reviews",
"200",
"--max-unit-price",
"4",
"--min-width",
"77",
"--require-prime",
"--delivery-by",
"tomorrow",
"--sort-by",
"price",
"--max-search-pages",
"3",
"--skip-details",
"--dry-run"
]);
assert.equal(request.query, "100w led bulbs");
assert.equal(request.filters.minRating, 4.5);
assert.equal(request.filters.minReviews, 200);
assert.equal(request.filters.maxUnitPrice, 4);
assert.equal(request.filters.minWidthInches, 77);
assert.equal(request.filters.requirePrime, true);
assert.equal(request.filters.deliveryBy, "tomorrow");
assert.equal(request.filters.sortBy, "price");
assert.equal(request.maxSearchPages, 3);
assert.equal(request.skipDetails, true);
assert.equal(request.dryRun, true);
});
it("maps output modes", () => {
assert.equal(parseCliRequest(["usb c cable", "--json"]).output, "json");
assert.equal(parseCliRequest(["usb c cable", "--markdown"]).output, "markdown");
assert.equal(parseCliRequest(["usb c cable", "--json", "--markdown"]).output, "both");
});
it("accepts max as a natural agent alias for limit", () => {
const request = parseCliRequest(["100w led bulbs", "--max", "5"]);
assert.equal(request.limit, 5);
});
it("normalizes natural-language filters for the target request", () => {
const request = parseCliRequest([
"100w led bulbs that cost less than $4 each and have over 200 reviews with a review score of more than 4.5 stars",
"--dry-run"
]);
assert.equal(request.query, "100w led bulbs");
assert.equal(request.filters.maxUnitPrice, 4);
assert.equal(request.filters.minReviews, 200);
assert.equal(request.filters.reviewCountComparison, "gt");
assert.equal(request.filters.minRating, 4.5);
assert.equal(request.filters.ratingComparison, "gt");
});
it("rejects limits below one", () => {
assert.throws(
() => parseCliRequest(["usb c cable", "--limit", "0"]),
/limit must be an integer greater than 0/
);
});
it("rejects unsafe large limits unless explicitly allowed", () => {
assert.throws(
() => parseCliRequest(["usb c cable", "--limit", "31"]),
/require --allow-large-limit/
);
});
it("rejects search page caps above five", () => {
assert.throws(
() => parseCliRequest(["usb c cable", "--max-search-pages", "6"]),
/max-search-pages must be 5 or less/
);
});
it("rejects unsupported sort modes", () => {
assert.throws(
() => parseCliRequest(["usb c cable", "--sort-by", "rating"]),
/sort-by must be either price or relevance/
);
});
it("builds the Amazon search URL without live network access", () => {
assert.equal(
buildSearchUrl("100w led bulbs"),
"https://www.amazon.com/s?k=100w%20led%20bulbs"
);
});
});
@@ -0,0 +1,118 @@
import assert from "node:assert/strict";
import { readFile } from "node:fs/promises";
import { join } from "node:path";
import { describe, it } from "node:test";
import { extractDetailPage } from "../src/detail-page.js";
const fixturePath = join(import.meta.dirname, "fixtures", "product-detail.html");
describe("extractDetailPage", () => {
it("extracts visible product detail fields from sanitized HTML", async () => {
const html = await readFile(fixturePath, "utf8");
const details = extractDetailPage(html, {
asin: "B0TESTLED1",
title: "Search title",
url: "https://www.amazon.com/dp/B0TESTLED1",
specs: [],
bullets: [],
matchedFilters: [],
missingFields: [],
extractionNotes: []
});
assert.equal(details.title, "Bright Daylight LED Bulbs 100W Equivalent, 50 Count");
assert.equal(details.price?.amount, 18.99);
assert.equal(details.delivery?.free, true);
assert.equal(details.availability, "In Stock");
assert.equal(details.seller, "Ships from Amazon.com");
assert.equal(details.bullets.length, 2);
assert.deepEqual(details.specs[0], { name: "Brand", value: "BrightCo" });
assert.equal(details.rating, 4.6);
assert.equal(details.reviewCount, 1234);
assert.equal(details.starBreakdown?.five, 72);
});
it("records missing detail-only fields", () => {
const details = extractDetailPage("<html><body><h1 id=\"productTitle\">Sparse Product</h1></body></html>", {
asin: "B0SPARSE01",
title: "Sparse",
url: "https://www.amazon.com/dp/B0SPARSE01",
specs: [],
bullets: [],
matchedFilters: [],
missingFields: [],
extractionNotes: []
});
assert.equal(details.price, undefined);
assert.ok(details.missingFields.includes("price"));
assert.ok(details.missingFields.includes("starBreakdown"));
});
it("drops script-like spec rows and trims availability metadata", () => {
const details = extractDetailPage(`
<h1 id="productTitle">Messy Product</h1>
<div id="availability">In Stock {"merchantId":"secretish"}</div>
<table>
<tr><td>Special Feature</td><td>(function(P) { tracking(); }) Real feature text</td></tr>
<tr><td>A19 Add to Cart logShoppableMetrics("x", true)</td><td>Buying Options</td></tr>
<tr><td>Wattage</td><td>15 watts</td></tr>
<tr><td>Customer Reviews</td><td>4.7 out of 5 stars tracking payload</td></tr>
</table>
`, {
asin: "B0MESSY001",
title: "Messy",
url: "https://www.amazon.com/dp/B0MESSY001",
specs: [],
bullets: [],
matchedFilters: [],
missingFields: [],
extractionNotes: []
});
assert.equal(details.availability, "In Stock");
assert.deepEqual(details.specs, [{ name: "Wattage", value: "15 watts" }]);
});
it("preserves a search-card Prime signal when detail delivery text omits Prime", () => {
const details = extractDetailPage(`
<h1 id="productTitle">Prime Sofa Bed</h1>
<div id="mir-layout-DELIVERY_BLOCK-slot-PRIMARY_DELIVERY_MESSAGE_LARGE">FREE delivery Tomorrow. Details</div>
<table>
<tr><td>Product Dimensions</td><td>35"D x 83"W x 31"H</td></tr>
</table>
`, {
asin: "B0PRIME123",
title: "Prime Sofa Bed",
url: "https://www.amazon.com/dp/B0PRIME123",
delivery: { display: "FREE delivery Tomorrow", free: true, prime: true },
specs: [],
bullets: [],
matchedFilters: [],
missingFields: [],
extractionNotes: []
});
assert.equal(details.delivery?.prime, true);
assert.equal(details.delivery?.free, true);
});
it("does not treat Prime in a detail title as Prime delivery", () => {
const details = extractDetailPage(`
<h1 id="productTitle">Prime Sofa Bed</h1>
<div id="mir-layout-DELIVERY_BLOCK-slot-PRIMARY_DELIVERY_MESSAGE_LARGE">FREE delivery Tomorrow. Details</div>
`, {
asin: "B0TITLE123",
title: "Prime Sofa Bed",
url: "https://www.amazon.com/dp/B0TITLE123",
specs: [],
bullets: [],
matchedFilters: [],
missingFields: [],
extractionNotes: []
});
assert.equal(details.delivery?.prime, false);
});
});
@@ -0,0 +1,126 @@
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import { applyFiltersAndLimit } from "../src/filters.js";
import type { ProductSearchResult } from "../src/types.js";
function product(overrides: Partial<ProductSearchResult>): ProductSearchResult {
return {
asin: "B0BASE0001",
title: "Base Product",
url: "https://www.amazon.com/dp/B0BASE0001",
specs: [],
bullets: [],
matchedFilters: [],
missingFields: [],
extractionNotes: [],
...overrides
};
}
describe("applyFiltersAndLimit", () => {
it("applies strict rating, review, and unit-price filters", () => {
const result = applyFiltersAndLimit([
product({
asin: "B0PASS0001",
rating: 4.6,
reviewCount: 201,
unitPrice: { amount: 3.99, currency: "USD", display: "$3.99/Count" }
}),
product({
asin: "B0FAIL0001",
rating: 4.5,
reviewCount: 200,
unitPrice: { amount: 3.99, currency: "USD", display: "$3.99/Count" }
}),
product({
asin: "B0UNKNOWN1",
rating: 4.7,
reviewCount: 300
})
], {
includeKeywords: [],
excludeKeywords: [],
minRating: 4.5,
ratingComparison: "gt",
minReviews: 200,
reviewCountComparison: "gt",
maxUnitPrice: 4
}, 10);
assert.deepEqual(result.results.map((item) => item.asin), ["B0PASS0001"]);
assert.equal(result.filteredOutCount, 2);
assert.match(result.filteredOutReasons["B0UNKNOWN1"]?.join(" ") ?? "", /unit price unknown/i);
});
it("sorts by rating, reviews, then price", () => {
const result = applyFiltersAndLimit([
product({ asin: "B0LOWPRICE", rating: 4.7, reviewCount: 1000, price: { amount: 15, currency: "USD", display: "$15.00" } }),
product({ asin: "B0HIGHRATE", rating: 4.9, reviewCount: 100, price: { amount: 40, currency: "USD", display: "$40.00" } }),
product({ asin: "B0MOREREV", rating: 4.7, reviewCount: 2000, price: { amount: 20, currency: "USD", display: "$20.00" } })
], { includeKeywords: [], excludeKeywords: [] }, 2);
assert.deepEqual(result.results.map((item) => item.asin), ["B0HIGHRATE", "B0MOREREV"]);
});
it("deduplicates repeated ASINs before limiting", () => {
const result = applyFiltersAndLimit([
product({ asin: "B0DUP0001", rating: 4.8, reviewCount: 1000 }),
product({ asin: "B0DUP0001", rating: 4.8, reviewCount: 1000 }),
product({ asin: "B0UNIQUE1", rating: 4.7, reviewCount: 900 })
], { includeKeywords: [], excludeKeywords: [] }, 10);
assert.deepEqual(result.results.map((item) => item.asin), ["B0DUP0001", "B0UNIQUE1"]);
});
it("applies width, Prime, and delivery-by filters", () => {
const result = applyFiltersAndLimit([
product({
asin: "B0MATCH001",
rating: 4.3,
reviewCount: 250,
price: { amount: 399, currency: "USD", display: "$399.00" },
delivery: { display: "FREE delivery Tomorrow", free: true, prime: true },
specs: [{ name: "Product Dimensions", value: "35\"D x 83\"W x 31\"H" }]
}),
product({
asin: "B0NOPRIME1",
rating: 4.5,
reviewCount: 300,
delivery: { display: "FREE delivery Tomorrow", free: true, prime: false },
specs: [{ name: "Product Dimensions", value: "35\"D x 84\"W x 31\"H" }]
}),
product({
asin: "B0NARROW01",
rating: 4.6,
reviewCount: 300,
delivery: { display: "FREE delivery Tomorrow", free: true, prime: true },
specs: [{ name: "Product Dimensions", value: "35\"D x 65\"W x 31\"H" }]
})
], {
includeKeywords: [],
excludeKeywords: [],
minRating: 4,
minReviews: 200,
minWidthInches: 77,
requirePrime: true,
deliveryBy: "tomorrow"
}, 10);
assert.deepEqual(result.results.map((item) => item.asin), ["B0MATCH001"]);
assert.match(result.filteredOutReasons["B0NOPRIME1"]?.join(" ") ?? "", /Prime delivery not verified/);
assert.match(result.filteredOutReasons["B0NARROW01"]?.join(" ") ?? "", /width 65/);
assert.ok(result.results[0]?.matchedFilters.includes("width >= 77 inches"));
assert.ok(result.results[0]?.matchedFilters.includes("Prime delivery"));
assert.ok(result.results[0]?.matchedFilters.includes("delivery by tomorrow"));
});
it("sorts by price when requested", () => {
const result = applyFiltersAndLimit([
product({ asin: "B0EXPENSIV", rating: 4.9, reviewCount: 1000, price: { amount: 500, currency: "USD", display: "$500.00" } }),
product({ asin: "B0CHEAPER1", rating: 4.1, reviewCount: 300, price: { amount: 200, currency: "USD", display: "$200.00" } })
], { includeKeywords: [], excludeKeywords: [], sortBy: "price" }, 10);
assert.deepEqual(result.results.map((item) => item.asin), ["B0CHEAPER1", "B0EXPENSIV"]);
});
});
+3
View File
@@ -0,0 +1,3 @@
# Fixtures
Fixtures in this directory are hand-crafted sanitized HTML snippets. They are not live Amazon snapshots and contain no cookies, account details, delivery location, scripts, tracking identifiers, or browser profile data.
@@ -0,0 +1,30 @@
<!-- Hand-crafted sanitized fixture. Not a live Amazon snapshot. -->
<html>
<body>
<h1 id="productTitle">Bright Daylight LED Bulbs 100W Equivalent, 50 Count</h1>
<span id="productTitle_feature_div"></span>
<span class="a-price"><span class="a-offscreen">$18.99</span></span>
<div id="mir-layout-DELIVERY_BLOCK-slot-PRIMARY_DELIVERY_MESSAGE_LARGE">FREE delivery Tomorrow</div>
<div id="availability">In Stock</div>
<div id="merchant-info">Ships from Amazon.com</div>
<div id="feature-bullets">
<ul>
<li><span>Energy efficient 100W equivalent bulbs.</span></li>
<li><span>Daylight color temperature for kitchens and garages.</span></li>
</ul>
</div>
<table id="productOverview_feature_div">
<tr><td>Brand</td><td>BrightCo</td></tr>
<tr><td>Light Type</td><td>LED</td></tr>
</table>
<span id="acrPopover" title="4.6 out of 5 stars"></span>
<span id="acrCustomerReviewText">1,234 ratings</span>
<table id="histogramTable">
<tr><td>5 star</td><td>72%</td></tr>
<tr><td>4 star</td><td>15%</td></tr>
<tr><td>3 star</td><td>7%</td></tr>
<tr><td>2 star</td><td>3%</td></tr>
<tr><td>1 star</td><td>3%</td></tr>
</table>
</body>
</html>
@@ -0,0 +1,23 @@
<!-- Hand-crafted sanitized fixture. Not a live Amazon snapshot. -->
<html>
<body>
<div data-component-type="s-search-result" data-asin="B0TESTLED1">
<h2><a class="a-link-normal s-line-clamp-2" href="/Bright-Daylight-Equivalent/dp/B0TESTLED1/ref=sr_1_1">Bright Daylight 100W Equivalent LED Bulbs, 50 Count</a></h2>
<span class="a-price"><span class="a-offscreen">$18.99</span></span>
<span class="a-size-base a-color-secondary">$0.38/Count</span>
<span aria-label="4.6 out of 5 stars"></span>
<a aria-label="1,234 ratings"></a>
<div class="a-row a-size-base a-color-secondary">FREE delivery Tomorrow</div>
<img class="s-image" src="https://m.media-amazon.com/images/I/test-led.jpg" />
</div>
<div data-component-type="s-search-result" data-asin="B0TESTLED2">
<span>Sponsored</span>
<h2><a href="https://www.amazon.com/gp/product/B0TESTLED2">Value LED Bulbs Soft White, Pack of 24</a></h2>
<span class="a-price"><span class="a-offscreen">$21.99</span></span>
<span aria-label="4.3 out of 5 stars"></span>
<a aria-label="543 ratings"></a>
<div>Delivery Friday</div>
</div>
<a class="s-pagination-next" href="/s?k=led+bulbs&amp;page=2">Next</a>
</body>
</html>
@@ -0,0 +1,83 @@
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import {
extractUnitCount,
parseMoney,
parseRating,
parseReviewCount,
parseStarBreakdown,
parseUnitPrice
} from "../src/parsers.js";
describe("parsers", () => {
it("parses USD money", () => {
assert.deepEqual(parseMoney("$19.99"), { amount: 19.99, currency: "USD", display: "$19.99" });
});
it("parses rating text", () => {
assert.equal(parseRating("4.6 out of 5 stars"), 4.6);
});
it("parses review count text", () => {
assert.equal(parseReviewCount("1,234 ratings"), 1234);
});
it("parses visible star histogram percentages", () => {
assert.deepEqual(parseStarBreakdown("5 star 72% 4 star 15% 3 star 7% 2 star 3% 1 star 3%"), {
five: 72,
four: 15,
three: 7,
two: 3,
one: 3,
basis: "percent"
});
});
it("extracts high-confidence unit counts", () => {
assert.deepEqual(extractUnitCount("LED bulbs, 100 Count, daylight"), {
count: 100,
confidence: "high",
source: "100 Count"
});
assert.deepEqual(extractUnitCount("Pack of 6 USB-C cables"), {
count: 6,
confidence: "high",
source: "Pack of 6"
});
});
it("distinguishes lower-confidence unit count phrases", () => {
assert.deepEqual(extractUnitCount("Set of 8 replacement filters"), {
count: 8,
confidence: "medium",
source: "Set of 8"
});
assert.deepEqual(extractUnitCount("6 bulbs soft white"), {
count: 6,
confidence: "low",
source: "6 bulbs"
});
});
it("parses visible unit prices", () => {
assert.deepEqual(parseUnitPrice("$0.33/Count"), {
amount: 0.33,
currency: "USD",
display: "$0.33/Count"
});
});
it("prefers the unit price when product price appears first", () => {
assert.deepEqual(parseUnitPrice("$9.99 ($5.00$5.00/count)"), {
amount: 5,
currency: "USD",
display: "$5.00/count"
});
});
it("parses whole-dollar and one-decimal prices", () => {
assert.deepEqual(parseMoney("$20"), { amount: 20, currency: "USD", display: "$20" });
assert.deepEqual(parseMoney("$19.9"), { amount: 19.9, currency: "USD", display: "$19.9" });
});
});
@@ -0,0 +1,56 @@
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import { extractWidthInches, formatWidthInches } from "../src/product-metrics.js";
import type { ProductSearchResult } from "../src/types.js";
function product(overrides: Partial<ProductSearchResult>): ProductSearchResult {
return {
asin: "B0WIDTH001",
title: "Base Product",
url: "https://www.amazon.com/dp/B0WIDTH001",
specs: [],
bullets: [],
matchedFilters: [],
missingFields: [],
extractionNotes: [],
...overrides
};
}
describe("product metrics", () => {
it("extracts explicit W dimensions from overall product specs", () => {
const width = extractWidthInches(product({
specs: [{ name: "Product Dimensions", value: "35\"D x 83.4\"W x 31\"H" }]
}));
assert.equal(width, 83.4);
});
it("uses dimension order labels when W is not repeated in the value", () => {
const width = extractWidthInches(product({
specs: [{ name: "Item Dimensions D x W x H", value: "35 x 108 x 31 inches" }]
}));
assert.equal(width, 108);
});
it("ignores non-overall width specs before falling back to title width", () => {
const width = extractWidthInches(product({
title: "83 Inch Sofa Bed",
specs: [
{ name: "Seat Interior Width", value: "65 Inches" },
{ name: "Arm Width", value: "5 Inches" },
{ name: "Minimum Required Door Width", value: "72 Inches" }
]
}));
assert.equal(width, 83);
});
it("formats unknown and decimal widths", () => {
assert.equal(formatWidthInches(undefined), "unknown");
assert.equal(formatWidthInches(83.4), "83.4\"");
assert.equal(formatWidthInches(108), "108\"");
});
});
@@ -0,0 +1,84 @@
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import { parseNaturalLanguageRequest } from "../src/query-parser.js";
describe("parseNaturalLanguageRequest", () => {
it("extracts the target LED bulb filters from natural language", () => {
const parsed = parseNaturalLanguageRequest(
"100w led bulbs that cost less than $4 each and have over 200 reviews with a review score of more than 4.5 stars"
);
assert.equal(parsed.query, "100w led bulbs");
assert.equal(parsed.filters.maxUnitPrice, 4);
assert.equal(parsed.filters.minReviews, 200);
assert.equal(parsed.filters.reviewCountComparison, "gt");
assert.equal(parsed.filters.minRating, 4.5);
assert.equal(parsed.filters.ratingComparison, "gt");
});
it("distinguishes inclusive review and rating phrasing", () => {
const parsed = parseNaturalLanguageRequest("usb c charger at least 500 reviews and 4.3 stars or better");
assert.equal(parsed.query, "usb c charger");
assert.equal(parsed.filters.minReviews, 500);
assert.equal(parsed.filters.reviewCountComparison, "gte");
assert.equal(parsed.filters.minRating, 4.3);
assert.equal(parsed.filters.ratingComparison, "gte");
});
it("cleans rating filter phrases from search query text", () => {
const parsed = parseNaturalLanguageRequest("usb c cable with over 1000 reviews and rating over 4 stars");
assert.equal(parsed.query, "usb c cable");
assert.equal(parsed.filters.minReviews, 1000);
assert.equal(parsed.filters.minRating, 4);
});
it("extracts rating filters without requiring the word of", () => {
const parsed = parseNaturalLanguageRequest(
"sofa bed, 77 inches or wider, over 50 reviews, rating 4.0 or better, Prime delivery only, sort by price low to high"
);
assert.equal(parsed.query, "sofa bed");
assert.equal(parsed.filters.minWidthInches, 77);
assert.equal(parsed.filters.widthComparison, "gte");
assert.equal(parsed.filters.minReviews, 50);
assert.equal(parsed.filters.reviewCountComparison, "gt");
assert.equal(parsed.filters.minRating, 4);
assert.equal(parsed.filters.ratingComparison, "gte");
assert.equal(parsed.filters.requirePrime, true);
assert.equal(parsed.filters.sortBy, "price");
});
it("extracts limit and max product price phrases", () => {
const parsed = parseNaturalLanguageRequest("return 5 wireless mouse under $30");
assert.equal(parsed.query, "wireless mouse");
assert.equal(parsed.limit, 5);
assert.equal(parsed.filters.maxPrice, 30);
});
it("extracts sofa width, Prime, and delivery urgency filters", () => {
const parsed = parseNaturalLanguageRequest(
"sofa bed of 77inches or wider in width, review score of 4 stars and higher, 200+ reviews and shipped with prime, color beige if possible, delivery by tomorrow"
);
assert.equal(parsed.query, "sofa bed color beige if possible");
assert.equal(parsed.filters.minWidthInches, 77);
assert.equal(parsed.filters.minRating, 4);
assert.equal(parsed.filters.ratingComparison, "gte");
assert.equal(parsed.filters.minReviews, 200);
assert.equal(parsed.filters.reviewCountComparison, "gte");
assert.equal(parsed.filters.requirePrime, true);
assert.equal(parsed.filters.deliveryBy, "tomorrow");
});
it("extracts overnight delivery requests", () => {
const parsed = parseNaturalLanguageRequest("queen sleeper sofa with overnight shipping and Prime");
assert.equal(parsed.query, "queen sleeper sofa");
assert.equal(parsed.filters.requirePrime, true);
assert.equal(parsed.filters.deliveryBy, "overnight");
});
});
@@ -0,0 +1,99 @@
import assert from "node:assert/strict";
import { describe, it } from "node:test";
import { createMarkdownReport, createResponse } from "../src/report.js";
describe("report", () => {
it("creates a structured JSON response", () => {
const response = createResponse({
query: "usb c cable",
filters: { includeKeywords: [], excludeKeywords: [], minReviews: 1000 },
limit: 1,
maxSearchPages: 2,
results: [],
filteredOutCount: 4,
warnings: ["partial extraction"],
now: () => new Date("2026-04-15T00:00:00.000Z")
});
assert.equal(response.source.site, "amazon.com");
assert.equal(response.filteredOutCount, 4);
assert.equal(response.source.scrapedAt, "2026-04-15T00:00:00.000Z");
});
it("creates concise markdown with product details and warnings", () => {
const markdown = createMarkdownReport(createResponse({
query: "usb c cable",
filters: { includeKeywords: [], excludeKeywords: [] },
limit: 1,
maxSearchPages: 2,
filteredOutCount: 0,
warnings: ["price missing for one item"],
now: () => new Date("2026-04-15T00:00:00.000Z"),
results: [{
asin: "B0TEST0001",
title: "USB-C Cable",
url: "https://www.amazon.com/dp/B0TEST0001",
price: { amount: 9.99, currency: "USD", display: "$9.99" },
rating: 4.7,
reviewCount: 1234,
delivery: { display: "FREE delivery Tomorrow", free: true },
specs: [{ name: "Length", value: "6 ft" }],
bullets: ["Braided cable"],
matchedFilters: [],
missingFields: ["starBreakdown"],
extractionNotes: []
}]
}));
assert.match(markdown, /USB-C Cable/);
assert.match(markdown, /\$9\.99/);
assert.match(markdown, /4\.7 stars/);
assert.match(markdown, /price missing/);
assert.match(markdown, /https:\/\/www\.amazon\.com\/dp\/B0TEST0001/);
});
it("creates a chat-safe template with direct product links and constraint status markers", () => {
const markdown = createMarkdownReport(createResponse({
query: "sofa bed beige",
filters: {
includeKeywords: [],
excludeKeywords: [],
minRating: 4,
minReviews: 200,
minWidthInches: 77,
requirePrime: true,
deliveryBy: "tomorrow"
},
limit: 10,
maxSearchPages: 2,
filteredOutCount: 3,
warnings: [],
now: () => new Date("2026-04-15T00:00:00.000Z"),
results: [{
asin: "B0SOFABED1",
title: "HONBAY Modular Sectional Sleeper",
url: "https://www.amazon.com/dp/B0SOFABED1",
price: { amount: 539.99, currency: "USD", display: "$539.99" },
rating: 4.1,
reviewCount: 242,
delivery: { display: "FREE delivery Tomorrow", free: true, prime: true },
specs: [{ name: "Item Dimensions D x W x H", value: "83.4\"D x 83.4\"W x 35\"H" }],
bullets: [],
matchedFilters: [],
missingFields: ["starBreakdown"],
extractionNotes: []
}]
}));
assert.match(markdown, /## Best Matches/);
assert.doesNotMatch(markdown, /\| # \| Product \| Price \| Rating \| Reviews \| Width \| Prime \| Delivery \| Link \|/);
assert.doesNotMatch(markdown, /\[Amazon\]\(https:\/\/www\.amazon\.com\/dp\/B0SOFABED1\)/);
assert.match(markdown, /HONBAY Modular Sectional Sleeper/);
assert.match(markdown, /Link: https:\/\/www\.amazon\.com\/dp\/B0SOFABED1/);
assert.match(markdown, /83\.4" OK/);
assert.match(markdown, /Prime OK/);
assert.match(markdown, /Tomorrow OK/);
assert.match(markdown, /Price: \$539\.99/);
});
});
@@ -0,0 +1,95 @@
import assert from "node:assert/strict";
import { readFile } from "node:fs/promises";
import { join } from "node:path";
import { describe, it } from "node:test";
import { extractSearchPage } from "../src/search-page.js";
const fixturePath = join(import.meta.dirname, "fixtures", "search-results.html");
describe("extractSearchPage", () => {
it("extracts normalized product candidates from sanitized search HTML", async () => {
const html = await readFile(fixturePath, "utf8");
const extracted = extractSearchPage(html, "https://www.amazon.com/s?k=led+bulbs");
assert.equal(extracted.status, "ok");
assert.equal(extracted.products.length, 2);
assert.equal(extracted.products[0]?.asin, "B0TESTLED1");
assert.equal(extracted.products[0]?.url, "https://www.amazon.com/dp/B0TESTLED1");
assert.equal(extracted.products[0]?.price?.amount, 18.99);
assert.equal(extracted.products[0]?.unitPrice?.amount, 0.38);
assert.equal(extracted.products[0]?.rating, 4.6);
assert.equal(extracted.products[0]?.reviewCount, 1234);
assert.equal(extracted.products[0]?.delivery?.free, true);
assert.equal(extracted.products[0]?.isSponsored, false);
assert.equal(extracted.products[1]?.isSponsored, true);
assert.equal(extracted.nextPageUrl, "https://www.amazon.com/s?k=led+bulbs&page=2");
});
it("detects Amazon challenge pages", () => {
const extracted = extractSearchPage("<html><title>Robot Check</title><body>Enter the characters you see below</body></html>", "https://www.amazon.com/s?k=x");
assert.equal(extracted.status, "challenge");
assert.match(extracted.warnings[0] ?? "", /challenge/i);
assert.equal(extracted.products.length, 0);
});
it("returns ok with no products for empty or cardless pages", () => {
const extracted = extractSearchPage("<html><body>No results</body></html>", "https://www.amazon.com/s?k=x");
assert.equal(extracted.status, "ok");
assert.deepEqual(extracted.products, []);
assert.equal(extracted.nextPageUrl, undefined);
});
it("skips malformed ASINs and cards without titles", () => {
const extracted = extractSearchPage(`
<div data-asin="bad"><h2><a href="/dp/bad">Bad ASIN</a></h2></div>
<div data-asin="B0VALID1234"></div>
`, "https://www.amazon.com/s?k=x");
assert.equal(extracted.status, "ok");
assert.equal(extracted.products.length, 0);
});
it("keeps candidates with missing price and records missing price later", () => {
const extracted = extractSearchPage(`
<div data-asin="B0NOPRICE1">
<h2><a href="/dp/B0NOPRICE1">No Price Product</a></h2>
</div>
`, "https://www.amazon.com/s?k=x");
assert.equal(extracted.products.length, 1);
assert.equal(extracted.products[0]?.price, undefined);
});
it("detects Prime badges even when visible delivery text omits the word Prime", () => {
const extracted = extractSearchPage(`
<div data-asin="B0PRIME123">
<h2><a href="/dp/B0PRIME123">Prime Sofa Bed</a></h2>
<span class="a-price"><span class="a-offscreen">$299.99</span></span>
<span aria-label="4.4 out of 5 stars"></span>
<span aria-label="246 ratings"></span>
<i class="a-icon a-icon-prime" aria-label="Amazon Prime"></i>
<span>FREE delivery Tomorrow</span>
</div>
`, "https://www.amazon.com/s?k=sofa+bed");
assert.equal(extracted.products.length, 1);
assert.equal(extracted.products[0]?.delivery?.prime, true);
assert.equal(extracted.products[0]?.delivery?.free, true);
assert.match(extracted.products[0]?.delivery?.display ?? "", /Tomorrow/);
});
it("does not treat Prime in a product title as Prime delivery", () => {
const extracted = extractSearchPage(`
<div data-asin="B0TITLE123">
<h2><a href="/dp/B0TITLE123">Prime Sofa Bed</a></h2>
<span>FREE delivery Tomorrow</span>
</div>
`, "https://www.amazon.com/s?k=sofa+bed");
assert.equal(extracted.products.length, 1);
assert.equal(extracted.products[0]?.delivery?.prime, false);
});
});
@@ -0,0 +1,46 @@
import assert from "node:assert/strict";
import { mkdtemp, mkdir, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { describe, it } from "node:test";
import { resolveWebAutomationRuntime } from "../src/web-automation-runtime.js";
async function createRuntime() {
const dir = await mkdtemp(join(tmpdir(), "amazon-shopping-runtime-"));
await writeFile(join(dir, "check-install.js"), "console.log('ok');\n");
await writeFile(join(dir, "package.json"), "{\"type\":\"module\"}\n");
await mkdir(join(dir, "node_modules", ".bin"), { recursive: true });
await writeFile(join(dir, "node_modules", ".bin", "tsx"), "#!/usr/bin/env node\n");
return dir;
}
describe("resolveWebAutomationRuntime", () => {
it("uses AMAZON_SHOPPING_WEB_AUTOMATION_DIR first", async () => {
const runtimeDir = await createRuntime();
const resolved = await resolveWebAutomationRuntime({
env: { AMAZON_SHOPPING_WEB_AUTOMATION_DIR: runtimeDir },
homeDir: "/missing-home",
skillDir: "/missing-skill"
});
assert.equal(resolved.scriptsDir, runtimeDir);
assert.deepEqual(resolved.checkInstall, {
cwd: runtimeDir,
command: "node",
args: ["check-install.js"]
});
});
it("returns a clear error when required files are missing", async () => {
const dir = await mkdtemp(join(tmpdir(), "amazon-shopping-runtime-missing-"));
await assert.rejects(
() => resolveWebAutomationRuntime({
env: { AMAZON_SHOPPING_WEB_AUTOMATION_DIR: dir },
homeDir: "/missing-home",
skillDir: "/missing-skill"
}),
/check-install.js/
);
});
});
+13
View File
@@ -0,0 +1,13 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"strict": true,
"esModuleInterop": true,
"forceConsistentCasingInFileNames": true,
"skipLibCheck": true,
"types": ["node"]
},
"include": ["src/**/*.ts", "tests/**/*.ts"]
}
+3
View File
@@ -0,0 +1,3 @@
node_modules/
dist/
*.log
+202
View File
@@ -0,0 +1,202 @@
---
name: ai-cli-dispatch
description: Dispatch AI CLI coding tasks to available clients (Codex, Claude Code, OpenCode) with automatic discovery, version checking, and execution.
metadata: {"clawdbot":{"emoji":"robot","requires":{"bins":["node"]}}}
---
# AI CLI Dispatch
Use this skill when the user wants to run a coding task through an AI CLI client such as Codex, Claude Code, or OpenCode.
The skill discovers installed clients, resolves versions, selects the best available tool, and forwards the task with arguments intact.
Use the local helper from the installed skill directory:
```bash
cd ~/.openclaw/workspace/skills/ai-cli-dispatch
scripts/ai-cli-dispatch --help
```
## Setup
```bash
cd ~/.openclaw/workspace/skills/ai-cli-dispatch
npm install
```
## Commands
### `list`
Discover and report all supported clients.
```bash
scripts/ai-cli-dispatch list --json
scripts/ai-cli-dispatch list --text
```
### `start` — start a background job
Starts a detached background job and returns immediately with a job ID.
```bash
scripts/ai-cli-dispatch start --client codex --prompt "refactor this function"
scripts/ai-cli-dispatch start --client claude --prompt "add tests for auth middleware"
```
### `run` — run a task (async by default)
Runs a prompt through a named client. By default this is **async**: it starts a background job and returns the job ID immediately. Use `--sync` to block until the client finishes and return the full result.
```bash
# Async (default) — returns job ID immediately
scripts/ai-cli-dispatch run --client codex --prompt "refactor this function"
scripts/ai-cli-dispatch run --client claude --prompt "add tests for auth middleware"
scripts/ai-cli-dispatch run --client opencode --prompt "migrate to ESM"
# Sync — blocks until completion and returns stdout/stderr/exitCode
scripts/ai-cli-dispatch run --client codex --prompt "fix lint errors" --sync
```
### `dispatch` — auto-resolve client and run (async by default)
Auto-resolves the client from prompt keywords, then executes. By default this is **async**. Use `--sync` to block until completion.
```bash
# Async (default)
scripts/ai-cli-dispatch dispatch "use codex to write tests"
scripts/ai-cli-dispatch dispatch "claude: add unit tests for utils.ts"
scripts/ai-cli-dispatch dispatch "opencode migrate to ESM"
# Sync
scripts/ai-cli-dispatch dispatch "review this PR" --client claude --sync
```
### Job lifecycle commands
After starting an async job, manage it with these subcommands:
```bash
# Check job status
scripts/ai-cli-dispatch status <job-id>
# Get results (only when status is completed)
scripts/ai-cli-dispatch results <job-id>
# Cancel a running job
scripts/ai-cli-dispatch cancel <job-id>
# List all jobs, newest first
scripts/ai-cli-dispatch list-jobs --json
scripts/ai-cli-dispatch list-jobs --status running --json
# Clean up old job files
scripts/ai-cli-dispatch cleanup-jobs --max-age 24h
```
## Async vs Sync Mode
The dispatcher is **async-first**: `run` and `dispatch` start a detached background job unless you pass `--sync`.
| Mode | Behavior | When to use |
|---|---|---|
| **Async** (default) | Starts a detached process, returns a `jobId` immediately, and stores output on disk. | Fire-and-forget tasks, long-running jobs, parallel dispatches, or when you need to poll/check results later. |
| **Sync** (`--sync`) | Blocks until the client subprocess exits, then returns `stdout`, `stderr`, and `exitCode` directly. | Short, interactive tasks where you need the result in the same turn. |
Use `--timeout <ms>` to control how long a job may run before it is terminated (default: 10 minutes / 600,000 ms for both async and sync). Use `--debug` to emit diagnostic metadata to stderr.
## Flags
| Flag | Description |
|---|---|
| `--sync` | Run synchronously and block until the client returns. Default is async (starts a background job). |
| `--timeout <ms>` | Timeout in milliseconds. Overrides the default and any config value. |
| `--debug` | Emit diagnostic JSON to stderr (command, args, PID, duration, exit signal). |
| `--json` | Output JSON (default). |
| `--text` | Output plain text instead of JSON. |
| `--client <name>` | Explicitly set the client (`codex`, `claude`, `opencode`). |
| `--prompt <text>` | The prompt to send to the client. |
| `--status <status>` | Filter `list-jobs` by status (`running`, `completed`, `failed`). |
| `--max-age <number>[h\|m\|s\|d]` | Maximum age for `cleanup-jobs` (default unit: hours). |
## Client Discovery
The skill searches for the following clients in order:
- `codex` — OpenAI Codex CLI
- `claude` — Anthropic Claude Code
- `opencode` — OpenCode CLI
Run `list` to see which clients are installed and their resolved versions.
## Job Lifecycle & Storage
Async jobs run as detached child processes. Each job writes a record to disk at:
```text
~/.openclaw/ai-cli-dispatch/jobs/<jobId>.json
```
A job moves through the following statuses:
| Status | Meaning |
|---|---|
| `running` | The client subprocess is active. |
| `completed` | The subprocess exited with code 0. |
| `failed` | The subprocess exited with a non-zero code. |
| `timed_out` | The job exceeded `--timeout` and was terminated. |
| `cancelled` | The job was cancelled via `cancel <job-id>`. |
Example async workflow:
```bash
# 1. Start a job
scripts/ai-cli-dispatch run --client codex --prompt "refactor auth module"
# → { "jobId": "a1b2c3d4...", "client": "codex", "status": "running" }
# 2. Poll status
scripts/ai-cli-dispatch status a1b2c3d4...
# → { "id": "a1b2c3d4...", "status": "running", ... }
# 3. Get results when done
scripts/ai-cli-dispatch results a1b2c3d4...
# → { "stdout": "...", "stderr": "...", "exitCode": 0, "client": "codex", "durationMs": 42000 }
```
## Background Jobs (Programmatic API)
For long-running or fire-and-forget tasks, use the programmatic job API:
```typescript
import { startJob, getJob, getJobResult, cancelJob, listJobs, cleanupJobs } from "./src/jobs.js";
// Start a detached job
const job = await startJob("codex", "refactor auth module", { timeoutMs: 300_000 });
console.log(job.id); // e.g. "a1b2c3d4..."
console.log(job.status); // "running"
// Poll for completion
const latest = getJob(job.id);
console.log(latest.status); // "running" | "completed" | "failed" | "timed_out" | "cancelled"
// Get result (throws if not completed)
const result = getJobResult(job.id);
console.log(result.stdout, result.exitCode);
// Cancel a running job
cancelJob(job.id);
// List all jobs (newest first)
const jobs = listJobs(); // Job[]
const running = listJobs({ filter: "running" });
// Clean up job files older than 24 hours (default)
cleanupJobs({ maxAgeMs: 24 * 60 * 60 * 1000 });
```
Job files include stdout, stderr, exit code, timing, and error state.
## Output Rules
- Normal JSON output redacts local file paths and credential metadata.
- Use `--debug` only when deeper troubleshooting requires internal paths and resolved config metadata.
+20
View File
@@ -0,0 +1,20 @@
{
"name": "ai-cli-dispatch",
"version": "1.0.0",
"description": "AI CLI dispatch tool for OpenClaw skills",
"type": "module",
"scripts": {
"ai-cli-dispatch": "tsx src/cli.ts",
"test": "node --import tsx --test tests/*.test.ts",
"typecheck": "tsc --noEmit"
},
"dependencies": {
"minimist": "^1.2.8"
},
"devDependencies": {
"@types/minimist": "^1.2.5",
"@types/node": "^24.8.1",
"tsx": "^4.20.6",
"typescript": "^5.9.3"
}
}
+26
View File
@@ -0,0 +1,26 @@
#!/usr/bin/env node
import { existsSync } from "node:fs";
import { dirname, join, resolve } from "node:path";
import { fileURLToPath } from "node:url";
import { spawnSync } from "node:child_process";
const scriptDir = dirname(fileURLToPath(import.meta.url));
const skillDir = resolve(scriptDir, "..");
const tsxBin = join(skillDir, "node_modules", ".bin", "tsx");
if (!existsSync(tsxBin)) {
process.stderr.write(`Missing local Node dependencies for ai-cli-dispatch. Run 'cd ${skillDir} && npm install' first.\n`);
process.exit(1);
}
const result = spawnSync(process.execPath, [tsxBin, join(skillDir, "src", "cli.ts"), ...process.argv.slice(2)], {
stdio: "inherit"
});
if (result.error) {
process.stderr.write(`${result.error.message}\n`);
process.exit(1);
}
process.exit(typeof result.status === "number" ? result.status : 1);
+85
View File
@@ -0,0 +1,85 @@
import type { ClientName, ExecResult, DebugInfo, Job } from "./types.js";
export interface RunContext {
jsonMode: boolean;
stdoutWrite: (chunk: string) => void;
stderrWrite: (chunk: string) => void;
}
export function reportError(err: unknown, jsonMode: boolean): number {
const message = err instanceof Error ? err.message : String(err);
if (jsonMode) {
console.error(JSON.stringify({ error: message }, null, 2));
} else {
console.error(message);
}
return 1;
}
export function reportCliError(message: string, jsonMode: boolean): number {
if (jsonMode) {
console.error(JSON.stringify({ error: message }, null, 2));
} else {
console.error(`Error: ${message}`);
}
return 1;
}
export async function handleSyncRun(
executePrompt: (
client: ClientName,
prompt: string,
options?: { timeoutMs?: number; debug?: boolean; onDebug?: (info: DebugInfo) => void }
) => Promise<ExecResult>,
client: ClientName,
prompt: string,
timeoutMs: number | undefined,
debug: boolean,
ctx: RunContext
): Promise<number> {
try {
const result = await executePrompt(client, prompt, {
timeoutMs,
debug,
onDebug: debug ? (info) => ctx.stderrWrite(JSON.stringify(info) + "\n") : undefined,
});
if (ctx.jsonMode) {
console.log(JSON.stringify(result, null, 2));
} else {
if (result.stdout) ctx.stdoutWrite(result.stdout);
if (result.stderr) ctx.stderrWrite(result.stderr);
}
return 0;
} catch (err) {
return reportError(err, ctx.jsonMode);
}
}
export async function handleAsyncRun(
startJob: (
client: ClientName,
prompt: string,
options?: { timeoutMs?: number; debug?: boolean; onDebug?: (info: DebugInfo) => void }
) => Promise<Job>,
client: ClientName,
prompt: string,
timeoutMs: number | undefined,
debug: boolean,
ctx: RunContext
): Promise<number> {
try {
const job = await startJob(client, prompt, {
timeoutMs,
debug,
onDebug: debug ? (info) => ctx.stderrWrite(JSON.stringify(info) + "\n") : undefined,
});
if (ctx.jsonMode) {
console.log(JSON.stringify({ jobId: job.id, client: job.client, status: job.status }, null, 2));
} else {
console.log(`Job ${job.id} started (${job.client}): ${job.status}`);
}
return 0;
} catch (err) {
return reportError(err, ctx.jsonMode);
}
}
+380
View File
@@ -0,0 +1,380 @@
import minimist from "minimist";
import { detectClients as realDetectClients } from "./detect.js";
import { executePrompt as realExecutePrompt } from "./execute.js";
import {
startJob as realStartJob,
getJob as realGetJob,
getJobResult as realGetJobResult,
cancelJob as realCancelJob,
listJobs as realListJobs,
cleanupJobs as realCleanupJobs,
} from "./jobs.js";
import { resolveClient as realResolveClient } from "./dispatch.js";
import { resolveConfig as realResolveConfig } from "./config.js";
import { CLIENT_NAMES } from "./constants.js";
import {
type ClientName,
type ClientInfo,
type ExecResult,
type DebugInfo,
type Job,
type JobStatus,
ClientNotFoundError,
} from "./types.js";
import {
reportError,
reportCliError,
handleSyncRun,
handleAsyncRun,
} from "./cli-helpers.js";
export interface CliDeps {
detectClients?: () => ClientInfo[];
executePrompt?: (
client: ClientName,
prompt: string,
options?: { timeoutMs?: number; debug?: boolean; onDebug?: (info: DebugInfo) => void }
) => Promise<ExecResult>;
startJob?: (
client: ClientName,
prompt: string,
options?: { timeoutMs?: number; debug?: boolean; onDebug?: (info: DebugInfo) => void }
) => Promise<Job>;
resolveClient?: (
prompt: string,
config?: { client?: ClientName; defaultClient?: ClientName }
) => ClientName | null;
resolveConfig?: () => { paths: Partial<Record<ClientName, string>>; defaultClient?: ClientName; timeout?: number };
stdoutWrite?: (chunk: string) => void;
stderrWrite?: (chunk: string) => void;
getJob?: (jobId: string) => Job;
getJobResult?: (jobId: string) => ExecResult;
cancelJob?: (jobId: string) => void;
listJobs?: (options?: { filter?: JobStatus }) => Job[];
cleanupJobs?: (options?: { maxAgeMs?: number }) => void;
}
function parseMaxAge(value: string): number | null {
const match = value.match(/^(\d+(?:\.\d+)?)\s*([hmsd]?)$/i);
if (!match) return null;
const num = parseFloat(match[1]);
if (!Number.isFinite(num) || num < 0) return null;
const unit = match[2].toLowerCase();
const multipliers: Record<string, number> = {
s: 1000,
m: 60 * 1000,
h: 60 * 60 * 1000,
d: 24 * 60 * 60 * 1000,
"": 60 * 60 * 1000,
};
const multiplier = multipliers[unit];
if (multiplier === undefined) return null;
return num * multiplier;
}
function printHelp(): void {
console.log(`AI CLI Dispatch
Usage:
ai-cli-dispatch list [--json|--text]
ai-cli-dispatch run --client <client> --prompt <prompt> [--sync] [--timeout <ms>] [--debug] [--json|--text]
ai-cli-dispatch dispatch <prompt> [--client <client>] [--sync] [--timeout <ms>] [--debug] [--json|--text]
ai-cli-dispatch start --client <client> --prompt <prompt> [--timeout <ms>] [--debug] [--json|--text]
ai-cli-dispatch status <job-id> [--json|--text]
ai-cli-dispatch results <job-id> [--json|--text]
ai-cli-dispatch cancel <job-id> [--json|--text]
ai-cli-dispatch list-jobs [--status running|completed|failed] [--json|--text]
ai-cli-dispatch cleanup-jobs [--max-age <number>[h|m|s|d]] [--json|--text]
ai-cli-dispatch --help
Flags:
--sync Run synchronously and block until the client returns (default is async)
--timeout Timeout in milliseconds (or override via config)
--debug Emit diagnostic JSON to stderr
--max-age Maximum age for cleanup (default unit: hours, e.g. 24h or 30m)
--status Filter jobs by status (running, completed, failed)
--json Output JSON (default)
--text Output plain text
Clients: codex, claude, opencode`);
}
export async function main(
argv: string[],
deps: CliDeps = {}
): Promise<number> {
const detectClients = deps.detectClients ?? realDetectClients;
const executePrompt = deps.executePrompt ?? realExecutePrompt;
const startJob = deps.startJob ?? realStartJob;
const resolveClient = deps.resolveClient ?? realResolveClient;
const resolveConfig = deps.resolveConfig ?? realResolveConfig;
const stdoutWrite = deps.stdoutWrite ?? ((c: string) => process.stdout.write(c));
const stderrWrite = deps.stderrWrite ?? ((c: string) => process.stderr.write(c));
const getJob = deps.getJob ?? realGetJob;
const getJobResult = deps.getJobResult ?? realGetJobResult;
const cancelJob = deps.cancelJob ?? realCancelJob;
const listJobs = deps.listJobs ?? realListJobs;
const cleanupJobs = deps.cleanupJobs ?? realCleanupJobs;
const rawArgs = argv.slice(2);
const parseArgs =
rawArgs[0]?.includes("cli.ts") ? rawArgs.slice(1) : rawArgs;
const args = minimist(parseArgs, {
string: ["client", "prompt", "timeout", "max-age"],
boolean: ["json", "text", "help", "debug", "sync"],
alias: { h: "help" },
});
const jsonMode = !args.text;
const debug = !!args.debug;
if (args.help) {
printHelp();
return 0;
}
const command = args._[0];
if (!command) {
printHelp();
return 1;
}
if (command === "list") {
const clients = detectClients();
if (jsonMode) {
console.log(JSON.stringify(clients, null, 2));
} else {
for (const c of clients) {
const status = c.found
? `${c.version ?? "unknown version"}`
: "✗ not found";
console.log(`${c.name}: ${status}`);
}
}
return 0;
}
if (command === "run") {
const client = args.client as ClientName | undefined;
const prompt = args.prompt as string | undefined;
if (!client || !CLIENT_NAMES.includes(client)) {
return reportCliError(
!client ? "--client is required" : `Unknown client: ${client}`,
jsonMode
);
}
if (!prompt) {
return reportCliError("--prompt is required", jsonMode);
}
const config = resolveConfig();
const parsedTimeout =
typeof args.timeout === "string" ? Number(args.timeout) : undefined;
const timeoutMs =
Number.isFinite(parsedTimeout) ? parsedTimeout : config.timeout;
const ctx = { jsonMode, stdoutWrite, stderrWrite };
if (args.sync) {
return await handleSyncRun(executePrompt, client, prompt, timeoutMs, debug, ctx);
}
return await handleAsyncRun(startJob, client, prompt, timeoutMs, debug, ctx);
}
if (command === "dispatch") {
let prompt = args.prompt as string | undefined;
if (!prompt && args._.length > 1) {
prompt = args._.slice(1).join(" ");
}
if (!prompt) {
return reportCliError("prompt is required", jsonMode);
}
const config = resolveConfig();
const explicitClient = args.client as ClientName | undefined;
const client = resolveClient(prompt, {
client: explicitClient,
defaultClient: config.defaultClient,
});
if (!client) {
return reportCliError("Could not resolve client from prompt", jsonMode);
}
const parsedTimeout =
typeof args.timeout === "string" ? Number(args.timeout) : undefined;
const timeoutMs =
Number.isFinite(parsedTimeout) ? parsedTimeout : config.timeout;
const ctx = { jsonMode, stdoutWrite, stderrWrite };
if (args.sync) {
return await handleSyncRun(executePrompt, client, prompt, timeoutMs, debug, ctx);
}
return await handleAsyncRun(startJob, client, prompt, timeoutMs, debug, ctx);
}
if (command === "start") {
const client = args.client as ClientName | undefined;
const prompt = args.prompt as string | undefined;
if (!client || !CLIENT_NAMES.includes(client)) {
return reportCliError(
!client ? "--client is required" : `Unknown client: ${client}`,
jsonMode
);
}
if (!prompt) {
return reportCliError("--prompt is required", jsonMode);
}
const config = resolveConfig();
const parsedTimeout =
typeof args.timeout === "string" ? Number(args.timeout) : undefined;
const timeoutMs =
Number.isFinite(parsedTimeout) ? parsedTimeout : config.timeout;
try {
const job = await startJob(client, prompt, {
timeoutMs,
debug,
onDebug: debug ? (info) => stderrWrite(JSON.stringify(info) + "\n") : undefined,
});
if (jsonMode) {
console.log(JSON.stringify({ jobId: job.id, client: job.client, status: job.status }, null, 2));
} else {
console.log(`Job ${job.id} started (${job.client}): ${job.status}`);
}
return 0;
} catch (err) {
return reportError(err, jsonMode);
}
}
if (command === "status") {
const jobId = args._[1];
if (!jobId) {
return reportCliError("job-id is required", jsonMode);
}
try {
const job = getJob(jobId);
if (jsonMode) {
console.log(JSON.stringify(job, null, 2));
} else {
console.log(`Job ${jobId}: ${job.status}`);
}
return 0;
} catch (err) {
return reportError(err, jsonMode);
}
}
if (command === "results") {
const jobId = args._[1];
if (!jobId) {
return reportCliError("job-id is required", jsonMode);
}
try {
const result = getJobResult(jobId);
if (jsonMode) {
console.log(JSON.stringify(result, null, 2));
} else {
if (result.stdout) stdoutWrite(result.stdout);
if (result.stderr) stderrWrite(result.stderr);
}
return 0;
} catch (err) {
return reportError(err, jsonMode);
}
}
if (command === "cancel") {
const jobId = args._[1];
if (!jobId) {
return reportCliError("job-id is required", jsonMode);
}
try {
const job = getJob(jobId);
if (job.status !== "running") {
return reportCliError(
`Job is not running (status: ${job.status})`,
jsonMode
);
}
cancelJob(jobId);
if (jsonMode) {
console.log(JSON.stringify({ jobId, cancelled: true }, null, 2));
} else {
console.log(`Job ${jobId} cancelled`);
}
return 0;
} catch (err) {
return reportError(err, jsonMode);
}
}
if (command === "list-jobs") {
try {
const filter = args.status as JobStatus | undefined;
const jobs = listJobs({ filter });
if (jsonMode) {
console.log(JSON.stringify(jobs, null, 2));
} else {
for (const job of jobs) {
console.log(`${job.id} (${job.client}): ${job.status}`);
}
}
return 0;
} catch (err) {
return reportError(err, jsonMode);
}
}
if (command === "cleanup-jobs") {
const maxAgeRaw = args["max-age"] as string | undefined;
let maxAgeMs: number | undefined;
if (maxAgeRaw !== undefined) {
const parsed = parseMaxAge(maxAgeRaw);
if (parsed === null) {
return reportCliError(
"Invalid --max-age format. Use: <number>[h|m|s|d], e.g. 24h",
jsonMode
);
}
maxAgeMs = parsed;
}
try {
const jobsBefore = listJobs();
cleanupJobs({ maxAgeMs });
const jobsAfter = listJobs();
const count = jobsBefore.length - jobsAfter.length;
if (jsonMode) {
console.log(JSON.stringify({ count }, null, 2));
} else {
console.log(`Cleaned ${count} jobs`);
}
return 0;
} catch (err) {
return reportError(err, jsonMode);
}
}
return reportError(`Unknown command: ${command}`, jsonMode);
}
const isMain =
import.meta.url.startsWith("file://") &&
!!process.argv[1] &&
import.meta.url.endsWith(process.argv[1].replace(/\\/g, "/"));
if (isMain) {
main(process.argv).then((code) => process.exit(code));
}
+111
View File
@@ -0,0 +1,111 @@
import { homedir } from "node:os";
import {
readFileSync as realReadFileSync,
existsSync as realExistsSync,
} from "node:fs";
import { spawnSync } from "node:child_process";
import type { ClientName } from "./types.js";
import { CLIENT_NAMES, isWindows } from "./constants.js";
export interface ResolvedConfig {
paths: Partial<Record<ClientName, string>>;
defaultClient?: ClientName;
timeout?: number;
}
export interface ResolveConfigOptions {
flags?: Record<string, string | boolean | undefined>;
env?: NodeJS.ProcessEnv;
homeDir?: string;
readFileSync?: (path: string, encoding: BufferEncoding) => string;
existsSync?: (path: string) => boolean;
whichSync?: (cmd: string) => string | undefined;
}
function defaultWhichSync(cmd: string): string | undefined {
const result = spawnSync(isWindows() ? "where" : "which", [cmd], {
encoding: "utf-8",
});
if (result.status === 0) {
return result.stdout.trim().split("\n")[0];
}
return undefined;
}
export function resolveConfig(
options: ResolveConfigOptions = {}
): ResolvedConfig {
const {
flags = {},
env = process.env,
homeDir = homedir(),
readFileSync = realReadFileSync,
existsSync = realExistsSync,
whichSync = defaultWhichSync,
} = options;
const configPath = `${homeDir}/.openclaw/ai-cli-dispatch.json`;
let fileConfig: Record<string, unknown> = {};
if (existsSync(configPath)) {
try {
fileConfig = JSON.parse(readFileSync(configPath, "utf-8"));
} catch {
fileConfig = {};
}
}
const filePaths = (fileConfig.paths ?? {}) as Partial<
Record<ClientName, string>
>;
const fileDefault = fileConfig.defaultClient as ClientName | undefined;
const paths: Partial<Record<ClientName, string>> = {};
for (const name of CLIENT_NAMES) {
const flagKey = `${name}-path`;
const envKey = `AI_CLI_${name.toUpperCase()}_PATH`;
const resolved =
(typeof flags[flagKey] === "string"
? (flags[flagKey] as string)
: undefined) ??
env[envKey] ??
filePaths[name] ??
whichSync(name);
if (resolved !== undefined) {
paths[name] = resolved;
}
}
const defaultClient =
(typeof flags["default-client"] === "string"
? (flags["default-client"] as string)
: undefined) ??
env.AI_CLI_DEFAULT_CLIENT ??
fileDefault;
const result: ResolvedConfig = { paths };
if (
defaultClient !== undefined &&
CLIENT_NAMES.includes(defaultClient as ClientName)
) {
result.defaultClient = defaultClient as ClientName;
}
const flagTimeout =
typeof flags.timeout === "string" ? Number(flags.timeout) : undefined;
const envTimeout =
typeof env.AI_CLI_TIMEOUT === "string"
? Number(env.AI_CLI_TIMEOUT)
: undefined;
const fileTimeout =
typeof fileConfig.timeout === "number" ? fileConfig.timeout : undefined;
const resolvedTimeout =
(Number.isFinite(flagTimeout) ? flagTimeout : undefined) ??
(Number.isFinite(envTimeout) ? envTimeout : undefined) ??
(Number.isFinite(fileTimeout) ? fileTimeout : undefined) ??
600_000;
result.timeout = resolvedTimeout;
return result;
}
+7
View File
@@ -0,0 +1,7 @@
import type { ClientName } from "./types.js";
export const CLIENT_NAMES: ClientName[] = ["codex", "claude", "opencode"];
export function isWindows(): boolean {
return process.platform === "win32";
}
+70
View File
@@ -0,0 +1,70 @@
import { spawnSync as defaultSpawnSync } from "node:child_process";
import { existsSync as defaultExistsSync } from "node:fs";
import { join } from "node:path";
import { type ClientName, type ClientInfo } from "./types.js";
import { CLIENT_NAMES, isWindows } from "./constants.js";
export interface DetectOptions {
pathEnv?: string;
spawnSync?: typeof defaultSpawnSync;
existsSync?: typeof defaultExistsSync;
}
function parseVersion(stdout: string): string | undefined {
const match = stdout.match(/v?\d+\.\d+\.\d+(?:[-+.]\w+)*/);
return match ? match[0].replace(/^v/, "") : undefined;
}
function findBinary(
name: string,
pathEnv: string,
spawnSyncImpl: typeof defaultSpawnSync,
existsSyncImpl: typeof defaultExistsSync
): string | undefined {
// Try system `which` / `where` first
const whichResult = spawnSyncImpl(isWindows() ? "where" : "which", [name], {
encoding: "utf-8",
env: { ...process.env, PATH: pathEnv },
});
if (whichResult.status === 0) {
const line = whichResult.stdout.trim().split("\n")[0];
if (line) return line;
}
// Fallback: walk PATH directories manually
const sep = isWindows() ? ";" : ":";
const ext = isWindows() ? ".exe" : "";
const dirs = pathEnv.split(sep);
for (const dir of dirs) {
if (!dir) continue;
const fullPath = join(dir, name + ext);
if (existsSyncImpl(fullPath)) {
return fullPath;
}
}
return undefined;
}
export function detectClients(options?: DetectOptions): ClientInfo[] {
const spawnSyncImpl = options?.spawnSync ?? defaultSpawnSync;
const existsSyncImpl = options?.existsSync ?? defaultExistsSync;
const pathEnv = options?.pathEnv ?? process.env.PATH ?? "";
return CLIENT_NAMES.map((name) => {
const path = findBinary(name, pathEnv, spawnSyncImpl, existsSyncImpl);
if (!path) {
return { name, found: false };
}
const versionResult = spawnSyncImpl(path, ["--version"], {
encoding: "utf-8",
});
const version =
versionResult.status === 0
? parseVersion(versionResult.stdout)
: undefined;
return { name, path, version, found: true };
});
}
+38
View File
@@ -0,0 +1,38 @@
import { CLIENT_NAMES } from "./constants.js";
import type { ClientName } from "./types.js";
export interface DispatchConfig {
defaultClient?: ClientName;
client?: ClientName;
}
export function resolveClient(
prompt: string,
config?: DispatchConfig
): ClientName | null {
// Explicit --client flag takes highest precedence
if (config?.client && CLIENT_NAMES.includes(config.client)) {
return config.client;
}
const lower = prompt.toLowerCase();
// Check for "open code" before "opencode" to handle the spaced variant
if (lower.includes("open code")) {
return "opencode";
}
if (lower.includes("claude")) {
return "claude";
}
if (lower.includes("codex")) {
return "codex";
}
if (lower.includes("opencode")) {
return "opencode";
}
return config?.defaultClient ?? null;
}
+164
View File
@@ -0,0 +1,164 @@
import { spawn as defaultSpawn } from "node:child_process";
import { existsSync as defaultExistsSync } from "node:fs";
import type { ClientName, ExecResult, DebugInfo, ExecuteOptions } from "./types.js";
import { ClientNotFoundError, ExecError } from "./types.js";
export const CLIENT_ARGS: Record<ClientName, (prompt: string) => string[]> = {
codex: (p) => ["exec", "--yolo", p],
claude: (p) => ["-p", p, "--dangerously-skip-permissions"],
opencode: (p) => ["run", "--dangerously-skip-permissions", p],
};
/**
* Known stderr noise patterns per client.
* When exit code is 0, lines matching these patterns are stripped from the
* returned stderr to prevent agents from misinterpreting informational
* diagnostics as errors. The raw (unfiltered) stderr is preserved in
* DebugInfo.rawStderr when --debug is active.
*/
const STDERR_NOISE_PATTERNS: Partial<Record<ClientName, RegExp[]>> = {
codex: [
/^\d{4}-\d{2}-\d{2}T[\d:.]+Z\s+ERROR\s+codex_core::util:\s+ReasoningSummary\w*\s+/,
],
};
function filterStderrNoise(client: ClientName, stderr: string, exitCode: number): string {
if (exitCode !== 0) return stderr;
const patterns = STDERR_NOISE_PATTERNS[client];
if (!patterns) return stderr;
const lines = stderr.split("\n");
const filtered = lines.filter((line) => !patterns.some((p) => p.test(line)));
return filtered.join("\n").replace(/\n+$/, "");
}
export async function executePrompt(
client: ClientName,
prompt: string,
options: ExecuteOptions = {}
): Promise<ExecResult> {
if (prompt.trim() === "") {
throw new ExecError("Prompt cannot be empty", {
stdout: "",
stderr: "",
exitCode: -1,
client,
durationMs: 0,
});
}
const spawnImpl = options.spawn ?? defaultSpawn;
const existsSyncImpl = options.existsSync ?? defaultExistsSync;
const timeoutMs = options.timeoutMs ?? 600_000;
const command = options.clientPath ?? client;
if (options.clientPath && !existsSyncImpl(options.clientPath)) {
throw new ClientNotFoundError(client);
}
const argBuilder = (CLIENT_ARGS as Record<string, (prompt: string) => string[]>)[client];
if (!argBuilder) {
throw new ExecError(`Unknown client: ${client}`, {
stdout: "",
stderr: "",
exitCode: -1,
client,
durationMs: 0,
});
}
const args = argBuilder(prompt);
return new Promise((resolve, reject) => {
let settled = false;
let timedOut = false;
let stdout = "";
let stderr = "";
let exitSignal: NodeJS.Signals | null = null;
const startMs = Date.now();
const child = spawnImpl(command, args, {
shell: false,
stdio: ["pipe", "pipe", "pipe"],
});
// Close stdin immediately so clients like codex don't hang waiting for input
child.stdin?.end();
child.stdout?.on("data", (chunk: Buffer | string) => {
stdout += chunk.toString();
});
child.stderr?.on("data", (chunk: Buffer | string) => {
stderr += chunk.toString();
});
const timeout = setTimeout(() => {
timedOut = true;
child.kill();
}, timeoutMs);
function settle(
err?: Error | undefined,
result?: ExecResult | undefined
): void {
if (settled) return;
settled = true;
clearTimeout(timeout);
const durationMs = Date.now() - startMs;
const rawStderr = stderr;
const cleanedStderr = filterStderrNoise(client, rawStderr, result?.exitCode ?? -1);
if (options.debug || options.onDebug) {
const effectiveExitCode = result?.exitCode ?? (err instanceof ExecError ? err.result.exitCode : null);
const debugInfo: DebugInfo = {
command,
args,
pid: child.pid ?? undefined,
exitCode: effectiveExitCode,
exitSignal,
durationMs,
stderrLength: rawStderr.length,
stdoutLength: stdout.length,
noisySuccess: effectiveExitCode === 0 && rawStderr.length > 0,
rawStderr: rawStderr !== cleanedStderr ? rawStderr : undefined,
};
options.onDebug?.(debugInfo);
}
if (err) {
if (err instanceof ExecError) {
err.result.stderr = cleanedStderr;
err.result.client = client;
err.result.durationMs = durationMs;
}
reject(err);
} else {
resolve({ ...result!, client, durationMs, stderr: cleanedStderr });
}
}
child.on("error", (err: NodeJS.ErrnoException) => {
if (err.code === "ENOENT") {
settle(new ClientNotFoundError(client));
} else {
settle(
new ExecError(err.message, { stdout, stderr, exitCode: -1, client, durationMs: 0 })
);
}
});
child.on("close", (code: number | null, signal: NodeJS.Signals | null) => {
exitSignal = signal;
if (timedOut) {
settle(
new ExecError(`Execution timed out after ${timeoutMs}ms`, {
stdout,
stderr,
exitCode: -1,
client,
durationMs: 0,
})
);
} else {
settle(undefined, { stdout, stderr, exitCode: code ?? -1, client, durationMs: 0 });
}
});
});
}
+128
View File
@@ -0,0 +1,128 @@
/**
* Job watcher — a tiny self-contained process that monitors a detached
* child and writes the final job record to disk.
*
* Invoked as: node --import tsx src/job-watcher.ts <jobFile> <command> <arg1> <arg2> ...
*
* The watcher is itself spawned as detached+unref'd by the CLI, so the CLI
* can return the job ID immediately while this process stays alive to capture
* the child's output and finalize the job file.
*/
import { spawn } from "node:child_process";
import { readFileSync, writeFileSync } from "node:fs";
import type { JobRecord, ExecResult, JobStatus, ClientName } from "./types.js";
// Must import CLIENT_ARGS to know the client command mapping
// And the noise filter for consistent stderr handling
/**
* Known stderr noise patterns per client (duplicated from execute.ts to keep
* the watcher self-contained with no runtime dependency on execute.ts).
*/
const STDERR_NOISE_PATTERNS: Partial<Record<ClientName, RegExp[]>> = {
codex: [
/^\d{4}-\d{2}-\d{2}T[\d:.]+Z\s+ERROR\s+codex_core::util:\s+ReasoningSummary\w*\s+/,
],
};
function filterStderrNoise(client: ClientName, stderr: string, exitCode: number): string {
if (exitCode !== 0) return stderr;
const patterns = STDERR_NOISE_PATTERNS[client];
if (!patterns) return stderr;
const lines = stderr.split("\n");
const filtered = lines.filter((line) => !patterns.some((p) => p.test(line)));
return filtered.join("\n").replace(/\n+$/, "");
}
const jobFile = process.argv[2];
const command = process.argv[3];
const childArgs = process.argv.slice(4);
if (!jobFile || !command) {
process.exit(1);
}
let record: JobRecord;
try {
record = JSON.parse(readFileSync(jobFile, "utf-8")) as JobRecord;
} catch {
process.exit(1);
}
const timeoutMs = 600_000; // 10 min default
let stdout = "";
let stderr = "";
let settled = false;
let timedOut = false;
const startMs = Date.now();
const child = spawn(command, childArgs, {
shell: false,
stdio: ["pipe", "pipe", "pipe"],
});
// Close stdin so clients like codex don't hang
child.stdin?.end();
// Update pid in job file
record.pid = child.pid ?? undefined;
writeFileSync(jobFile, JSON.stringify(record, null, 2));
child.stdout?.on("data", (chunk: Buffer | string) => {
stdout += chunk.toString();
});
child.stderr?.on("data", (chunk: Buffer | string) => {
stderr += chunk.toString();
});
const timeout = setTimeout(() => {
timedOut = true;
try { child.kill("SIGTERM"); } catch { /* ignore */ }
}, timeoutMs);
function finalize(status: JobStatus, result?: ExecResult, error?: string) {
if (settled) return;
settled = true;
clearTimeout(timeout);
const completedAt = new Date().toISOString();
const durationMs = Date.now() - startMs;
const finalRecord: JobRecord = {
...record,
status,
stdout,
stderr: result ? filterStderrNoise(record.client, stderr, result.exitCode) : stderr,
result: result ? { ...result, durationMs } : undefined,
error,
completedAt,
};
try {
// Check if the job was cancelled while we were running
const current = JSON.parse(readFileSync(jobFile, "utf-8")) as JobRecord;
if (current.status === "cancelled") {
return; // Don't overwrite a cancelled job
}
writeFileSync(jobFile, JSON.stringify(finalRecord, null, 2));
} catch { /* best effort */ }
process.exit(0);
}
child.on("error", (err: NodeJS.ErrnoException) => {
finalize("failed", undefined, err.message);
});
child.on("close", (code: number | null, signal: NodeJS.Signals | null) => {
if (timedOut || signal) {
finalize("timed_out", {
stdout, stderr, exitCode: -1, client: record.client, durationMs: 0,
});
} else if (code !== null && code !== 0) {
finalize("failed", {
stdout, stderr, exitCode: code, client: record.client, durationMs: 0,
});
} else {
finalize("completed", {
stdout, stderr, exitCode: code ?? 0, client: record.client, durationMs: 0,
});
}
});
+290
View File
@@ -0,0 +1,290 @@
import { spawn as defaultSpawn } from "node:child_process";
import { existsSync as defaultExistsSync, mkdirSync as defaultMkdirSync, readFileSync as defaultReadFileSync, readdirSync as defaultReaddirSync, statSync as defaultStatSync, unlinkSync as defaultUnlinkSync, writeFileSync as defaultWriteFileSync } from "node:fs";
import { randomUUID } from "node:crypto";
import { CLIENT_ARGS } from "./execute.js";
import type { ClientName, Job, JobRecord, JobStartOptions, ExecResult, JobStatus } from "./types.js";
import { JobNotFoundError, JobResultUnavailableError } from "./types.js";
export interface JobOperationsOptions {
jobDir?: string;
fs?: {
mkdirSync: typeof defaultMkdirSync;
writeFileSync: typeof defaultWriteFileSync;
readFileSync: typeof defaultReadFileSync;
readdirSync: typeof defaultReaddirSync;
existsSync: typeof defaultExistsSync;
statSync: typeof defaultStatSync;
unlinkSync: typeof defaultUnlinkSync;
};
}
export interface StartJobOptions extends JobStartOptions, JobOperationsOptions {
/** Override the watcher spawn (for testing). When provided, startJob calls
* this instead of spawning `node --import tsx job-watcher.ts ...` */
spawnWatcher?: (jobFilePath: string, command: string, args: string[]) => { pid?: number; unref?: () => void };
}
const DEFAULT_JOB_DIR = `${process.env.HOME || process.env.USERPROFILE}/.openclaw/ai-cli-dispatch/jobs`;
function getJobDir(options?: { jobDir?: string }): string {
return options?.jobDir ?? DEFAULT_JOB_DIR;
}
function writeJobFile(
jobDir: string,
record: JobRecord,
fs: NonNullable<JobOperationsOptions["fs"]>
): void {
fs.mkdirSync(jobDir, { recursive: true });
fs.writeFileSync(`${jobDir}/${record.id}.json`, JSON.stringify(record, null, 2));
}
function readJobFile(
jobId: string,
jobDir: string,
fs: NonNullable<JobOperationsOptions["fs"]>
): JobRecord {
const path = `${jobDir}/${jobId}.json`;
try {
return JSON.parse(fs.readFileSync(path, "utf-8")) as JobRecord;
} catch (err: any) {
if (err.code === "ENOENT") {
throw new JobNotFoundError(jobId);
}
throw err;
}
}
export async function startJob(
client: ClientName,
prompt: string,
options: StartJobOptions = {}
): Promise<Job> {
const jobId = randomUUID();
const jobDir = getJobDir(options);
const fs = options.fs ?? {
mkdirSync: defaultMkdirSync,
writeFileSync: defaultWriteFileSync,
readFileSync: defaultReadFileSync,
readdirSync: defaultReaddirSync,
existsSync: defaultExistsSync,
statSync: defaultStatSync,
unlinkSync: defaultUnlinkSync,
};
const spawnImpl = options.spawn ?? defaultSpawn;
const argBuilder = (CLIENT_ARGS as Record<string, (prompt: string) => string[]>)[client];
if (!argBuilder) {
const startedAt = new Date().toISOString();
const errRecord: JobRecord = {
id: jobId,
client,
prompt,
status: "failed",
startedAt,
completedAt: startedAt,
stdout: "",
stderr: "",
error: `Unknown client: ${client}`,
};
writeJobFile(jobDir, errRecord, fs);
return { id: jobId, client, prompt, status: "failed", startedAt, error: errRecord.error };
}
const clientArgs = argBuilder(prompt);
const command = options.clientPath ?? client;
const startedAt = new Date().toISOString();
const record: JobRecord = {
id: jobId,
client,
prompt,
status: "running",
startedAt,
pid: undefined,
stdout: "",
stderr: "",
};
writeJobFile(jobDir, record, fs);
// Spawn a companion watcher process that outlives this CLI invocation.
// The watcher monitors the actual client (codex/claude/opencode), captures
// stdout/stderr, and writes the final job record to disk on exit.
// This allows the CLI to return the job ID immediately while the watcher
// stays alive to finalize the job.
let watcher: { pid?: number; unref?: () => void };
if (options.spawnWatcher) {
// Test path: use the injected watcher mock
watcher = options.spawnWatcher(`${jobDir}/${jobId}.json`, command, clientArgs);
} else {
const watcherArgs = [
"--import", "tsx",
new URL("./job-watcher.ts", import.meta.url).pathname,
`${jobDir}/${jobId}.json`,
command,
...clientArgs,
];
watcher = spawnImpl("node", watcherArgs, {
detached: true,
shell: false,
stdio: "ignore",
});
watcher.unref?.();
}
// Give the watcher a tick to spawn and record the real child PID
await new Promise((r) => setTimeout(r, 100));
// Re-read the job file to pick up the watcher's PID update
let updatedRecord: JobRecord;
try {
updatedRecord = JSON.parse(fs.readFileSync(`${jobDir}/${jobId}.json`, "utf-8")) as JobRecord;
} catch {
updatedRecord = record;
}
return {
id: jobId,
client,
prompt,
status: "running",
startedAt,
pid: updatedRecord.pid ?? watcher.pid ?? undefined,
};
}
export function getJob(jobId: string, options: JobOperationsOptions = {}): Job {
const jobDir = getJobDir(options);
const fs = options.fs ?? {
mkdirSync: defaultMkdirSync,
writeFileSync: defaultWriteFileSync,
readFileSync: defaultReadFileSync,
readdirSync: defaultReaddirSync,
existsSync: defaultExistsSync,
statSync: defaultStatSync,
unlinkSync: defaultUnlinkSync,
};
const record = readJobFile(jobId, jobDir, fs);
const { stdout: _stdout, stderr: _stderr, ...job } = record;
return job;
}
export function getJobResult(jobId: string, options: JobOperationsOptions = {}): ExecResult {
const job = getJob(jobId, options);
if (job.status !== "completed") {
throw new JobResultUnavailableError(jobId, job.status);
}
if (!job.result) {
throw new JobResultUnavailableError(jobId, "completed");
}
return job.result;
}
export function cancelJob(jobId: string, options: JobOperationsOptions = {}): void {
const jobDir = getJobDir(options);
const fs = options.fs ?? {
mkdirSync: defaultMkdirSync,
writeFileSync: defaultWriteFileSync,
readFileSync: defaultReadFileSync,
readdirSync: defaultReaddirSync,
existsSync: defaultExistsSync,
statSync: defaultStatSync,
unlinkSync: defaultUnlinkSync,
};
const record = readJobFile(jobId, jobDir, fs);
if (record.status !== "running") {
return;
}
// Kill the client child process (PID recorded by the watcher)
if (record.pid) {
try {
process.kill(record.pid, "SIGTERM");
} catch {
// ignore — process may have already exited
}
}
// Update the job file to cancelled
const cancelledRecord: JobRecord = {
...record,
status: "cancelled",
completedAt: new Date().toISOString(),
};
writeJobFile(jobDir, cancelledRecord, fs);
}
export function listJobs(options: JobOperationsOptions & { filter?: JobStatus } = {}): Job[] {
const jobDir = getJobDir(options);
const fs = options.fs ?? {
mkdirSync: defaultMkdirSync,
writeFileSync: defaultWriteFileSync,
readFileSync: defaultReadFileSync,
readdirSync: defaultReaddirSync,
existsSync: defaultExistsSync,
statSync: defaultStatSync,
unlinkSync: defaultUnlinkSync,
};
if (!fs.existsSync(jobDir)) {
return [];
}
const entries = fs.readdirSync(jobDir).filter((f) => f.endsWith(".json"));
const jobs: Job[] = [];
for (const entry of entries) {
const jobId = entry.replace(/\.json$/, "");
try {
const record = readJobFile(jobId, jobDir, fs);
const { stdout: _stdout, stderr: _stderr, ...job } = record;
jobs.push(job);
} catch {
// ignore corrupt/missing files
}
}
jobs.sort((a, b) => new Date(b.startedAt).getTime() - new Date(a.startedAt).getTime());
if (options.filter) {
return jobs.filter((j) => j.status === options.filter);
}
return jobs;
}
export function cleanupJobs(options: JobOperationsOptions & { maxAgeMs?: number } = {}): void {
const jobDir = getJobDir(options);
const fs = options.fs ?? {
mkdirSync: defaultMkdirSync,
writeFileSync: defaultWriteFileSync,
readFileSync: defaultReadFileSync,
readdirSync: defaultReaddirSync,
existsSync: defaultExistsSync,
statSync: defaultStatSync,
unlinkSync: defaultUnlinkSync,
};
const maxAgeMs = options.maxAgeMs ?? 24 * 60 * 60 * 1000;
if (!fs.existsSync(jobDir)) {
return;
}
const now = Date.now();
const entries = fs.readdirSync(jobDir).filter((f) => f.endsWith(".json"));
for (const entry of entries) {
const path = `${jobDir}/${entry}`;
try {
const stat = fs.statSync(path);
if (now - stat.mtimeMs > maxAgeMs) {
fs.unlinkSync(path);
}
} catch {
// ignore
}
}
}
+100
View File
@@ -0,0 +1,100 @@
import type { ChildProcess } from "node:child_process";
import type { PathLike } from "node:fs";
export type ClientName = "codex" | "claude" | "opencode";
export interface ClientInfo {
name: ClientName;
path?: string;
version?: string;
found: boolean;
}
export interface ExecResult {
stdout: string;
stderr: string;
exitCode: number;
client: ClientName;
durationMs: number;
}
export interface DebugInfo {
command: string;
args: string[];
pid?: number;
exitCode: number | null;
exitSignal: NodeJS.Signals | null;
durationMs: number;
stderrLength: number;
stdoutLength: number;
noisySuccess: boolean;
/** Unfiltered stderr before noise removal (only present when --debug). */
rawStderr?: string;
}
export interface ExecuteOptions {
clientPath?: string;
timeoutMs?: number;
debug?: boolean;
onDebug?: (info: DebugInfo) => void;
spawn?: (command: string, args: string[], options?: { shell?: boolean }) => ChildProcess;
existsSync?: (path: PathLike) => boolean;
}
export type JobStatus = "running" | "completed" | "failed" | "timed_out" | "cancelled";
export interface Job {
id: string;
client: ClientName;
prompt: string;
status: JobStatus;
result?: ExecResult;
error?: string;
startedAt: string;
completedAt?: string;
pid?: number;
}
/**
* On-disk storage contract for job files under
* ~/.openclaw/ai-cli-dispatch/jobs/<jobId>.json
*/
export interface JobRecord extends Job {
stdout: string;
stderr: string;
}
export interface JobStartOptions extends ExecuteOptions {
jobDir?: string;
}
export class ClientNotFoundError extends Error {
constructor(client: string) {
super(`Client "${client}" not found or not installed.`);
this.name = "ClientNotFoundError";
}
}
export class ExecError extends Error {
readonly result: ExecResult;
constructor(message: string, result: ExecResult) {
super(message);
this.name = "ExecError";
this.result = result;
}
}
export class JobNotFoundError extends Error {
constructor(jobId: string) {
super(`Job "${jobId}" not found.`);
this.name = "JobNotFoundError";
}
}
export class JobResultUnavailableError extends Error {
constructor(jobId: string, status: JobStatus) {
super(`Job "${jobId}" result is not available (status: ${status}).`);
this.name = "JobResultUnavailableError";
}
}
@@ -0,0 +1,278 @@
import { describe, it } from "node:test";
import assert from "node:assert";
import {
reportError,
reportCliError,
handleSyncRun,
handleAsyncRun,
} from "../src/cli-helpers.js";
import type { ExecResult, Job } from "../src/types.js";
function captureConsole() {
const logs: string[] = [];
const errors: string[] = [];
const origLog = console.log;
const origError = console.error;
console.log = (...args: unknown[]) => logs.push(args.map(String).join(" "));
console.error = (...args: unknown[]) => errors.push(args.map(String).join(" "));
return {
logs,
errors,
restore() {
console.log = origLog;
console.error = origError;
},
};
}
describe("reportError", () => {
it("prints JSON error for Error instance when jsonMode=true", () => {
const out = captureConsole();
try {
const code = reportError(new Error("boom"), true);
assert.strictEqual(code, 1);
assert.strictEqual(out.errors.length, 1);
const parsed = JSON.parse(out.errors[0]);
assert.strictEqual(parsed.error, "boom");
} finally {
out.restore();
}
});
it("prints plain text error for Error instance when jsonMode=false", () => {
const out = captureConsole();
try {
const code = reportError(new Error("boom"), false);
assert.strictEqual(code, 1);
assert.strictEqual(out.errors.length, 1);
assert.strictEqual(out.errors[0], "boom");
} finally {
out.restore();
}
});
it("prints JSON error for non-Error value when jsonMode=true", () => {
const out = captureConsole();
try {
const code = reportError("plain string", true);
assert.strictEqual(code, 1);
const parsed = JSON.parse(out.errors[0]);
assert.strictEqual(parsed.error, "plain string");
} finally {
out.restore();
}
});
});
describe("reportCliError", () => {
it("prints JSON error with jsonMode=true", () => {
const out = captureConsole();
try {
const code = reportCliError("missing arg", true);
assert.strictEqual(code, 1);
const parsed = JSON.parse(out.errors[0]);
assert.strictEqual(parsed.error, "missing arg");
} finally {
out.restore();
}
});
it("prints prefixed text error with jsonMode=false", () => {
const out = captureConsole();
try {
const code = reportCliError("missing arg", false);
assert.strictEqual(code, 1);
assert.strictEqual(out.errors[0], "Error: missing arg");
} finally {
out.restore();
}
});
});
describe("handleSyncRun", () => {
it("returns 0 and prints JSON result in jsonMode", async () => {
const out = captureConsole();
try {
const result: ExecResult = {
stdout: "out",
stderr: "err",
exitCode: 0,
client: "codex",
durationMs: 10,
};
const code = await handleSyncRun(
async () => result,
"codex",
"hello",
5000,
false,
{ jsonMode: true, stdoutWrite: () => {}, stderrWrite: () => {} }
);
assert.strictEqual(code, 0);
assert.strictEqual(out.logs.length, 1);
const parsed = JSON.parse(out.logs[0]);
assert.strictEqual(parsed.stdout, "out");
} finally {
out.restore();
}
});
it("returns 0 and writes stdout/stderr in text mode", async () => {
const out = captureConsole();
const stdout: string[] = [];
const stderr: string[] = [];
try {
const result: ExecResult = {
stdout: "out",
stderr: "err",
exitCode: 0,
client: "codex",
durationMs: 10,
};
const code = await handleSyncRun(
async () => result,
"codex",
"hello",
5000,
false,
{
jsonMode: false,
stdoutWrite: (c) => stdout.push(c),
stderrWrite: (c) => stderr.push(c),
}
);
assert.strictEqual(code, 0);
assert.strictEqual(stdout.join(""), "out");
assert.strictEqual(stderr.join(""), "err");
assert.strictEqual(out.logs.length, 0);
} finally {
out.restore();
}
});
it("passes timeoutMs and debug to executePrompt", async () => {
const out = captureConsole();
try {
let received: any;
const code = await handleSyncRun(
async (_c, _p, opts) => {
received = opts;
return {
stdout: "",
stderr: "",
exitCode: 0,
client: "codex",
durationMs: 1,
};
},
"codex",
"hello",
12345,
true,
{ jsonMode: true, stdoutWrite: () => {}, stderrWrite: () => {} }
);
assert.strictEqual(code, 0);
assert.strictEqual(received?.timeoutMs, 12345);
assert.strictEqual(received?.debug, true);
} finally {
out.restore();
}
});
it("returns 1 and prints error JSON when executePrompt throws", async () => {
const out = captureConsole();
try {
const code = await handleSyncRun(
async () => {
throw new Error("fail");
},
"codex",
"hello",
undefined,
false,
{ jsonMode: true, stdoutWrite: () => {}, stderrWrite: () => {} }
);
assert.strictEqual(code, 1);
const parsed = JSON.parse(out.errors[0]);
assert.strictEqual(parsed.error, "fail");
} finally {
out.restore();
}
});
});
describe("handleAsyncRun", () => {
it("returns 0 and prints JSON job in jsonMode", async () => {
const out = captureConsole();
try {
const job: Job = {
id: "j1",
client: "codex",
prompt: "hi",
status: "running",
startedAt: "2024-01-01T00:00:00Z",
};
const code = await handleAsyncRun(
async () => job,
"codex",
"hi",
undefined,
false,
{ jsonMode: true, stdoutWrite: () => {}, stderrWrite: () => {} }
);
assert.strictEqual(code, 0);
const parsed = JSON.parse(out.logs[0]);
assert.strictEqual(parsed.jobId, "j1");
assert.strictEqual(parsed.status, "running");
} finally {
out.restore();
}
});
it("returns 0 and prints text job info", async () => {
const out = captureConsole();
try {
const job: Job = {
id: "j1",
client: "codex",
prompt: "hi",
status: "running",
startedAt: "2024-01-01T00:00:00Z",
};
const code = await handleAsyncRun(
async () => job,
"codex",
"hi",
undefined,
false,
{ jsonMode: false, stdoutWrite: () => {}, stderrWrite: () => {} }
);
assert.strictEqual(code, 0);
assert.ok(out.logs[0].includes("j1"));
assert.ok(out.logs[0].includes("running"));
} finally {
out.restore();
}
});
it("returns 1 and prints error JSON when startJob throws", async () => {
const out = captureConsole();
try {
const code = await handleAsyncRun(
async () => {
throw new Error("fail");
},
"codex",
"hi",
undefined,
false,
{ jsonMode: true, stdoutWrite: () => {}, stderrWrite: () => {} }
);
assert.strictEqual(code, 1);
const parsed = JSON.parse(out.errors[0]);
assert.strictEqual(parsed.error, "fail");
} finally {
out.restore();
}
});
});
File diff suppressed because it is too large Load Diff
+205
View File
@@ -0,0 +1,205 @@
import { describe, it } from "node:test";
import assert from "node:assert";
import { resolveConfig } from "../src/config.js";
describe("resolveConfig", () => {
it("returns empty config when no sources are present", () => {
const config = resolveConfig({
existsSync: () => false,
whichSync: () => undefined,
});
assert.deepStrictEqual(config.paths, {});
assert.strictEqual(config.defaultClient, undefined);
});
it("loads config from file", () => {
const config = resolveConfig({
existsSync: () => true,
readFileSync: () =>
JSON.stringify({
paths: { codex: "/file/codex", claude: "/file/claude" },
defaultClient: "claude",
}),
whichSync: () => undefined,
});
assert.strictEqual(config.paths.codex, "/file/codex");
assert.strictEqual(config.paths.claude, "/file/claude");
assert.strictEqual(config.defaultClient, "claude");
});
it("overrides file config with env vars", () => {
const config = resolveConfig({
env: { AI_CLI_CODEX_PATH: "/env/codex" },
existsSync: () => true,
readFileSync: () => JSON.stringify({ paths: { codex: "/file/codex" } }),
whichSync: () => undefined,
});
assert.strictEqual(config.paths.codex, "/env/codex");
});
it("overrides env vars with CLI flags", () => {
const config = resolveConfig({
flags: { "codex-path": "/flag/codex" },
env: { AI_CLI_CODEX_PATH: "/env/codex" },
existsSync: () => true,
readFileSync: () => JSON.stringify({ paths: { codex: "/file/codex" } }),
whichSync: () => undefined,
});
assert.strictEqual(config.paths.codex, "/flag/codex");
});
it("falls back to PATH detection when config file is missing", () => {
const config = resolveConfig({
existsSync: () => false,
whichSync: (cmd) =>
cmd === "opencode" ? "/usr/bin/opencode" : undefined,
});
assert.strictEqual(config.paths.opencode, "/usr/bin/opencode");
assert.strictEqual(config.paths.codex, undefined);
assert.strictEqual(config.paths.claude, undefined);
});
it("respects full priority ordering: flag > env > file > PATH", () => {
const config = resolveConfig({
flags: {
"codex-path": "/flag/codex",
"claude-path": "/flag/claude",
"opencode-path": "/flag/opencode",
"default-client": "opencode",
},
env: {
AI_CLI_CODEX_PATH: "/env/codex",
AI_CLI_CLAUDE_PATH: "/env/claude",
AI_CLI_OPENCODE_PATH: "/env/opencode",
AI_CLI_DEFAULT_CLIENT: "claude",
},
existsSync: () => true,
readFileSync: () =>
JSON.stringify({
paths: {
codex: "/file/codex",
claude: "/file/claude",
opencode: "/file/opencode",
},
defaultClient: "codex",
}),
whichSync: (cmd) => `/path/${cmd}`,
});
assert.strictEqual(config.paths.codex, "/flag/codex");
assert.strictEqual(config.paths.claude, "/flag/claude");
assert.strictEqual(config.paths.opencode, "/flag/opencode");
assert.strictEqual(config.defaultClient, "opencode");
});
it("uses env var for default client when no flag is given", () => {
const config = resolveConfig({
env: { AI_CLI_DEFAULT_CLIENT: "claude" },
existsSync: () => false,
whichSync: () => undefined,
});
assert.strictEqual(config.defaultClient, "claude");
});
it("uses file default client when no env var is given", () => {
const config = resolveConfig({
existsSync: () => true,
readFileSync: () => JSON.stringify({ defaultClient: "codex" }),
whichSync: () => undefined,
});
assert.strictEqual(config.defaultClient, "codex");
});
it("ignores invalid defaultClient from file", () => {
const config = resolveConfig({
existsSync: () => true,
readFileSync: () => JSON.stringify({ defaultClient: "foo" }),
whichSync: () => undefined,
});
assert.strictEqual(config.defaultClient, undefined);
});
it("ignores invalid defaultClient from env var", () => {
const config = resolveConfig({
env: { AI_CLI_DEFAULT_CLIENT: "bar" },
existsSync: () => false,
whichSync: () => undefined,
});
assert.strictEqual(config.defaultClient, undefined);
});
it("ignores invalid defaultClient from flag", () => {
const config = resolveConfig({
flags: { "default-client": "baz" },
existsSync: () => false,
whichSync: () => undefined,
});
assert.strictEqual(config.defaultClient, undefined);
});
it("returns default timeout of 600000 when no sources are present", () => {
const config = resolveConfig({
existsSync: () => false,
whichSync: () => undefined,
});
assert.strictEqual(config.timeout, 600_000);
});
it("loads timeout from file config", () => {
const config = resolveConfig({
existsSync: () => true,
readFileSync: () => JSON.stringify({ timeout: 120_000 }),
whichSync: () => undefined,
});
assert.strictEqual(config.timeout, 120_000);
});
it("overrides file timeout with env var", () => {
const config = resolveConfig({
env: { AI_CLI_TIMEOUT: "240000" },
existsSync: () => true,
readFileSync: () => JSON.stringify({ timeout: 120_000 }),
whichSync: () => undefined,
});
assert.strictEqual(config.timeout, 240_000);
});
it("overrides env timeout with CLI flag", () => {
const config = resolveConfig({
flags: { timeout: "480000" },
env: { AI_CLI_TIMEOUT: "240000" },
existsSync: () => true,
readFileSync: () => JSON.stringify({ timeout: 120_000 }),
whichSync: () => undefined,
});
assert.strictEqual(config.timeout, 480_000);
});
it("respects full priority ordering for timeout: flag > env > file > default", () => {
const config = resolveConfig({
flags: { timeout: "480000" },
env: { AI_CLI_TIMEOUT: "240000" },
existsSync: () => true,
readFileSync: () => JSON.stringify({ timeout: 120_000 }),
whichSync: () => undefined,
});
assert.strictEqual(config.timeout, 480_000);
});
it("ignores invalid timeout from env var and falls back to default", () => {
const config = resolveConfig({
env: { AI_CLI_TIMEOUT: "not-a-number" },
existsSync: () => false,
whichSync: () => undefined,
});
assert.strictEqual(config.timeout, 600_000);
});
it("ignores invalid timeout from file and falls back to default", () => {
const config = resolveConfig({
existsSync: () => true,
readFileSync: () => JSON.stringify({ timeout: "not-a-number" }),
whichSync: () => undefined,
});
assert.strictEqual(config.timeout, 600_000);
});
});
+144
View File
@@ -0,0 +1,144 @@
import { describe, it } from "node:test";
import assert from "node:assert";
import { detectClients } from "../src/detect.js";
function mockSpawnSync(
responses: Map<string, { status: number; stdout: string }>
): any {
return (cmd: string, args: string[], _opts: unknown) => {
const key = [cmd, ...args].join(" ");
const hit = responses.get(key);
if (hit) {
return { status: hit.status, stdout: hit.stdout, stderr: "" };
}
return { status: 1, stdout: "", stderr: "" };
};
}
function mockExistsSync(
allowedPaths: Set<string>
): any {
return (p: string) => allowedPaths.has(p);
}
describe("detectClients", () => {
it("returns all three clients when found on PATH", () => {
const responses = new Map([
["which codex", { status: 0, stdout: "/usr/local/bin/codex\n" }],
["which claude", { status: 0, stdout: "/usr/local/bin/claude\n" }],
["which opencode", { status: 0, stdout: "/usr/local/bin/opencode\n" }],
["/usr/local/bin/codex --version", { status: 0, stdout: "1.0.0\n" }],
["/usr/local/bin/claude --version", { status: 0, stdout: "1.0.0\n" }],
["/usr/local/bin/opencode --version", { status: 0, stdout: "1.0.0\n" }],
]);
const clients = detectClients({
pathEnv: "/usr/local/bin",
spawnSync: mockSpawnSync(responses),
existsSync: () => false,
});
assert.strictEqual(clients.length, 3);
for (const c of clients) {
assert.strictEqual(c.found, true);
assert.ok(c.path?.includes(c.name));
assert.strictEqual(c.version, "1.0.0");
}
});
it("returns found: false for a missing client", () => {
const responses = new Map([
["which codex", { status: 0, stdout: "/usr/local/bin/codex\n" }],
["which claude", { status: 0, stdout: "/usr/local/bin/claude\n" }],
["which opencode", { status: 1, stdout: "" }],
["/usr/local/bin/codex --version", { status: 0, stdout: "1.0.0\n" }],
["/usr/local/bin/claude --version", { status: 0, stdout: "1.0.0\n" }],
]);
const clients = detectClients({
pathEnv: "/usr/local/bin",
spawnSync: mockSpawnSync(responses),
existsSync: () => false,
});
assert.strictEqual(clients.length, 3);
const opencode = clients.find((c) => c.name === "opencode");
assert.ok(opencode);
assert.strictEqual(opencode!.found, false);
assert.strictEqual(opencode!.path, undefined);
assert.strictEqual(opencode!.version, undefined);
const codex = clients.find((c) => c.name === "codex");
assert.ok(codex);
assert.strictEqual(codex!.found, true);
});
it("returns found: false for all when none are on PATH", () => {
const responses = new Map<string, { status: number; stdout: string }>();
const clients = detectClients({
pathEnv: "/usr/local/bin",
spawnSync: mockSpawnSync(responses),
existsSync: () => false,
});
assert.strictEqual(clients.length, 3);
for (const c of clients) {
assert.strictEqual(c.found, false);
assert.strictEqual(c.path, undefined);
assert.strictEqual(c.version, undefined);
}
});
it("handles PATH with duplicate entries via fallback walk", () => {
const responses = new Map<string, { status: number; stdout: string }>();
// which fails for all, so fallback walk is used
const allowedPaths = new Set([
"/usr/bin/codex",
"/usr/bin/claude",
"/usr/bin/opencode",
]);
const clients = detectClients({
pathEnv: "/usr/bin:/usr/bin:/usr/local/bin",
spawnSync: mockSpawnSync(responses),
existsSync: mockExistsSync(allowedPaths),
});
assert.strictEqual(clients.length, 3);
for (const c of clients) {
assert.strictEqual(c.found, true);
assert.strictEqual(c.path, `/usr/bin/${c.name}`);
}
});
it("parses version string from noisy output", () => {
const responses = new Map([
["which codex", { status: 0, stdout: "/opt/bin/codex\n" }],
["which claude", { status: 0, stdout: "/opt/bin/claude\n" }],
["which opencode", { status: 0, stdout: "/opt/bin/opencode\n" }],
[
"/opt/bin/codex --version",
{ status: 0, stdout: "codex version 1.2.3 (build abc)\n" },
],
[
"/opt/bin/claude --version",
{ status: 0, stdout: "Claude Code 0.4.5-beta\n" },
],
["/opt/bin/opencode --version", { status: 0, stdout: "v2.0.0\n" }],
]);
const clients = detectClients({
pathEnv: "/opt/bin",
spawnSync: mockSpawnSync(responses),
existsSync: () => false,
});
assert.strictEqual(clients.find((c) => c.name === "codex")!.version, "1.2.3");
assert.strictEqual(
clients.find((c) => c.name === "claude")!.version,
"0.4.5-beta"
);
assert.strictEqual(clients.find((c) => c.name === "opencode")!.version, "2.0.0");
});
});
@@ -0,0 +1,83 @@
import { describe, it } from "node:test";
import assert from "node:assert";
import { resolveClient } from "../src/dispatch.js";
import type { ClientName } from "../src/types.js";
describe("resolveClient", () => {
it('returns "codex" when prompt contains "use codex"', () => {
const result = resolveClient("use codex to refactor this");
assert.strictEqual(result, "codex");
});
it('returns "claude" when prompt contains "tell claude to..."', () => {
const result = resolveClient("tell claude to review my code");
assert.strictEqual(result, "claude");
});
it('returns "opencode" when prompt contains "run with opencode"', () => {
const result = resolveClient("run with opencode");
assert.strictEqual(result, "opencode");
});
it('returns "opencode" when prompt contains "open code"', () => {
const result = resolveClient("open code this file");
assert.strictEqual(result, "opencode");
});
it("returns null when no keyword matches and no default is configured", () => {
const result = resolveClient("hello world");
assert.strictEqual(result, null);
});
it("returns defaultClient when no keyword matches", () => {
const result = resolveClient("hello world", { defaultClient: "claude" });
assert.strictEqual(result, "claude");
});
it("prefers --client flag over keyword parsing", () => {
const result = resolveClient("use codex for this", { client: "claude" });
assert.strictEqual(result, "claude");
});
it("prefers --client flag over defaultClient", () => {
const result = resolveClient("hello world", {
client: "opencode",
defaultClient: "codex",
});
assert.strictEqual(result, "opencode");
});
it("handles uppercase CODEX", () => {
const result = resolveClient("Use CODEX please");
assert.strictEqual(result, "codex");
});
it("handles mixed-case Claude", () => {
const result = resolveClient("Tell Claude to fix this");
assert.strictEqual(result, "claude");
});
it("returns first match when multiple clients are mentioned", () => {
const result = resolveClient("ask claude or codex to help");
assert.strictEqual(result, "claude");
});
it("returns null for empty prompt with no default", () => {
const result = resolveClient("");
assert.strictEqual(result, null);
});
it("returns defaultClient for empty prompt", () => {
const result = resolveClient("", { defaultClient: "codex" });
assert.strictEqual(result, "codex");
});
it("validates --client flag value", () => {
// Invalid client flag should fall back to keyword/default behavior
const result = resolveClient("use codex", {
client: "invalid" as ClientName,
});
// If client flag is invalid, we should fall back to keyword matching
assert.strictEqual(result, "codex");
});
});
+444
View File
@@ -0,0 +1,444 @@
import { describe, it } from "node:test";
import assert from "node:assert";
import { EventEmitter } from "node:events";
import { Readable } from "node:stream";
import { executePrompt } from "../src/execute.js";
import { ClientNotFoundError, ExecError } from "../src/types.js";
import type { ClientName } from "../src/types.js";
interface MockScenario {
stdout?: string;
stderr?: string;
exitCode?: number;
error?: NodeJS.ErrnoException;
hang?: boolean;
}
function createMockChildProcess(scenario: MockScenario): any {
const child = new EventEmitter() as any;
child.pid = 12345;
child.stdout = Readable.from(
scenario.stdout !== undefined ? [scenario.stdout] : []
);
child.stderr = Readable.from(
scenario.stderr !== undefined ? [scenario.stderr] : []
);
child.killed = false;
child.kill = () => {
child.killed = true;
process.nextTick(() => {
child.emit("exit", null, "SIGTERM");
child.emit("close", null, "SIGTERM");
});
return true;
};
let stdoutEnded = scenario.stdout === undefined;
let stderrEnded = scenario.stderr === undefined;
child.stdout.on("end", () => {
stdoutEnded = true;
maybeClose();
});
child.stderr.on("end", () => {
stderrEnded = true;
maybeClose();
});
function maybeClose() {
if (stdoutEnded && stderrEnded && !scenario.hang) {
child.emit("exit", scenario.exitCode ?? 0, null);
child.emit("close", scenario.exitCode ?? 0, null);
}
}
process.nextTick(() => {
if (scenario.error) {
child.emit("error", scenario.error);
}
});
return child;
}
function mockSpawn(
scenarios: Map<string, MockScenario>
): any {
return (cmd: string, args: string[], _opts: any): any => {
const key = [cmd, ...args].join(" ");
const scenario = scenarios.get(key);
if (!scenario) {
return createMockChildProcess({
error: Object.assign(new Error("spawn ENOENT"), { code: "ENOENT" }),
});
}
return createMockChildProcess(scenario);
};
}
function mockExistsSync(allowedPaths: Set<string>): any {
return (p: string) => allowedPaths.has(p);
}
describe("executePrompt", () => {
it("captures stdout for a successful codex execution", async () => {
const scenarios = new Map<string, MockScenario>([
['codex exec --yolo "hello world"', { stdout: "result\n", exitCode: 0 }],
]);
const result = await executePrompt("codex", '"hello world"', {
spawn: mockSpawn(scenarios),
existsSync: () => true,
});
assert.strictEqual(result.stdout, "result\n");
assert.strictEqual(result.stderr, "");
assert.strictEqual(result.exitCode, 0);
});
it("captures stderr for a successful claude execution", async () => {
const scenarios = new Map<string, MockScenario>([
["claude -p hello --dangerously-skip-permissions", { stdout: "", stderr: "warning\n", exitCode: 0 }],
]);
const result = await executePrompt("claude", "hello", {
spawn: mockSpawn(scenarios),
existsSync: () => true,
});
assert.strictEqual(result.stdout, "");
assert.strictEqual(result.stderr, "warning\n");
assert.strictEqual(result.exitCode, 0);
});
it("returns non-zero exit code without throwing", async () => {
const scenarios = new Map<string, MockScenario>([
["opencode run --dangerously-skip-permissions fail", { stdout: "", stderr: "error\n", exitCode: 1 }],
]);
const result = await executePrompt("opencode", "fail", {
spawn: mockSpawn(scenarios),
existsSync: () => true,
});
assert.strictEqual(result.exitCode, 1);
assert.strictEqual(result.stderr, "error\n");
});
it("throws ClientNotFoundError when binary emits ENOENT", async () => {
const scenarios = new Map<string, MockScenario>();
await assert.rejects(
executePrompt("codex", "hello", {
spawn: mockSpawn(scenarios),
existsSync: () => true,
}),
(err: unknown) => err instanceof ClientNotFoundError
);
});
it("throws ClientNotFoundError when explicit clientPath does not exist", async () => {
await assert.rejects(
executePrompt("claude", "hello", {
clientPath: "/nonexistent/claude",
existsSync: mockExistsSync(new Set()),
}),
(err: unknown) => err instanceof ClientNotFoundError
);
});
it("rejects with ExecError when timeout is exceeded", async () => {
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo slow", { hang: true }],
]);
await assert.rejects(
executePrompt("codex", "slow", {
spawn: mockSpawn(scenarios),
existsSync: () => true,
timeoutMs: 10,
}),
(err: unknown) =>
err instanceof ExecError &&
err.message.includes("timed out") &&
err.result.exitCode === -1
);
});
it("passes prompts with special characters unchanged", async () => {
const scenarios = new Map<string, MockScenario>([
[
'codex exec --yolo "quotes\nnewlines"',
{ stdout: "ok", exitCode: 0 },
],
]);
const result = await executePrompt("codex", '"quotes\nnewlines"', {
spawn: mockSpawn(scenarios),
existsSync: () => true,
});
assert.strictEqual(result.stdout, "ok");
assert.strictEqual(result.exitCode, 0);
});
it("rejects empty prompt", async () => {
await assert.rejects(
executePrompt("codex", "", {
spawn: mockSpawn(new Map()),
existsSync: () => true,
}),
(err: unknown) =>
err instanceof ExecError && err.message.includes("empty")
);
});
it("rejects whitespace-only prompt", async () => {
await assert.rejects(
executePrompt("claude", " ", {
spawn: mockSpawn(new Map()),
existsSync: () => true,
}),
(err: unknown) =>
err instanceof ExecError && err.message.includes("empty")
);
});
it("throws ExecError for invalid client at runtime", async () => {
await assert.rejects(
executePrompt("bogus" as unknown as ClientName, "hello", {
spawn: mockSpawn(new Map()),
existsSync: () => true,
}),
(err: unknown) =>
err instanceof ExecError && err.message.includes("Unknown client")
);
});
it("includes client and durationMs in result", async () => {
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo hello", { stdout: "ok", exitCode: 0 }],
]);
const result = await executePrompt("codex", "hello", {
spawn: mockSpawn(scenarios),
existsSync: () => true,
});
assert.strictEqual(result.client, "codex");
assert.strictEqual(typeof result.durationMs, "number");
assert.ok(result.durationMs >= 0);
});
it("rejects with ExecError containing custom timeout value", async () => {
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo slow", { hang: true }],
]);
await assert.rejects(
executePrompt("codex", "slow", {
spawn: mockSpawn(scenarios),
existsSync: () => true,
timeoutMs: 50,
}),
(err: unknown) =>
err instanceof ExecError &&
err.message === "Execution timed out after 50ms" &&
err.result.exitCode === -1 &&
err.result.client === "codex" &&
typeof err.result.durationMs === "number"
);
});
it("uses default timeout of 600000 when timeoutMs is not provided", async () => {
const delays: number[] = [];
const origSetTimeout = global.setTimeout;
(global as any).setTimeout = function(callback: any, delay: number) {
delays.push(delay);
return origSetTimeout(callback, delay);
};
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo hello", { stdout: "ok", exitCode: 0 }],
]);
try {
await executePrompt("codex", "hello", {
spawn: mockSpawn(scenarios),
existsSync: () => true,
});
assert.strictEqual(delays[0], 600_000);
} finally {
global.setTimeout = origSetTimeout;
}
});
it("emits debug info via onDebug when debug is true for successful execution", async () => {
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo hello", { stdout: "ok", stderr: "warn", exitCode: 0 }],
]);
const debugInfos: any[] = [];
const result = await executePrompt("codex", "hello", {
spawn: mockSpawn(scenarios),
existsSync: () => true,
debug: true,
onDebug: (info) => debugInfos.push(info),
});
assert.strictEqual(result.exitCode, 0);
assert.strictEqual(debugInfos.length, 1);
const info = debugInfos[0];
assert.strictEqual(info.command, "codex");
assert.deepStrictEqual(info.args, ["exec", "--yolo", "hello"]);
assert.strictEqual(info.pid, 12345);
assert.strictEqual(info.exitCode, 0);
assert.strictEqual(info.exitSignal, null);
assert.strictEqual(info.stderrLength, 4);
assert.strictEqual(info.stdoutLength, 2);
assert.strictEqual(typeof info.durationMs, "number");
assert.ok(info.durationMs >= 0);
});
it("emits debug info via onDebug when debug is true for failed execution", async () => {
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo fail", { stdout: "", stderr: "error", exitCode: 1 }],
]);
const debugInfos: any[] = [];
const result = await executePrompt("codex", "fail", {
spawn: mockSpawn(scenarios),
existsSync: () => true,
debug: true,
onDebug: (info) => debugInfos.push(info),
});
assert.strictEqual(result.exitCode, 1);
assert.strictEqual(debugInfos.length, 1);
assert.strictEqual(debugInfos[0].exitCode, 1);
assert.strictEqual(debugInfos[0].stderrLength, 5);
assert.strictEqual(debugInfos[0].stdoutLength, 0);
});
it("emits debug info via onDebug for spawn errors", async () => {
const scenarios = new Map<string, MockScenario>();
const debugInfos: any[] = [];
await assert.rejects(
executePrompt("codex", "hello", {
spawn: mockSpawn(scenarios),
existsSync: () => true,
debug: true,
onDebug: (info) => debugInfos.push(info),
}),
(err: unknown) => err instanceof ClientNotFoundError
);
assert.strictEqual(debugInfos.length, 1);
assert.strictEqual(debugInfos[0].command, "codex");
assert.deepStrictEqual(debugInfos[0].args, ["exec", "--yolo", "hello"]);
assert.strictEqual(debugInfos[0].exitCode, null);
assert.strictEqual(debugInfos[0].exitSignal, null);
});
it("reports noisySuccess=true when stderr is non-empty and exitCode is 0", async () => {
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo hello", { stdout: "ok", stderr: "warn", exitCode: 0 }],
]);
const debugInfos: any[] = [];
await executePrompt("codex", "hello", {
spawn: mockSpawn(scenarios),
existsSync: () => true,
debug: true,
onDebug: (info) => debugInfos.push(info),
});
assert.strictEqual(debugInfos[0].noisySuccess, true);
});
it("reports noisySuccess=false when stderr is empty and exitCode is 0", async () => {
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo hello", { stdout: "ok", stderr: "", exitCode: 0 }],
]);
const debugInfos: any[] = [];
await executePrompt("codex", "hello", {
spawn: mockSpawn(scenarios),
existsSync: () => true,
debug: true,
onDebug: (info) => debugInfos.push(info),
});
assert.strictEqual(debugInfos[0].noisySuccess, false);
});
it("reports noisySuccess=false when exitCode is non-zero even if stderr is non-empty", async () => {
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo fail", { stdout: "", stderr: "error", exitCode: 1 }],
]);
const debugInfos: any[] = [];
await executePrompt("codex", "fail", {
spawn: mockSpawn(scenarios),
existsSync: () => true,
debug: true,
onDebug: (info) => debugInfos.push(info),
});
assert.strictEqual(debugInfos[0].noisySuccess, false);
});
it("filters codex ReasoningSummary noise from stderr on exit code 0", async () => {
const noisyStderr = [
'2026-05-20T18:33:01.969310Z ERROR codex_core::util: ReasoningSummaryPartAdded without active item',
'2026-05-20T18:33:03.281713Z ERROR codex_core::util: ReasoningSummaryDelta without active item',
'2026-05-20T18:33:03.348247Z ERROR codex_core::util: ReasoningSummaryDelta without active item',
].join('\n');
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo hello", { stdout: "Hello world!", stderr: noisyStderr, exitCode: 0 }],
]);
const result = await executePrompt("codex", "hello", {
spawn: mockSpawn(scenarios),
existsSync: () => true,
});
assert.strictEqual(result.exitCode, 0);
assert.strictEqual(result.stderr, "");
assert.strictEqual(result.stdout, "Hello world!");
});
it("preserves real error stderr from codex on non-zero exit code", async () => {
const noisyStderr = [
'2026-05-20T18:33:01.969310Z ERROR codex_core::util: ReasoningSummaryDelta without active item',
'Error: something actually went wrong',
].join('\n');
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo fail", { stdout: "", stderr: noisyStderr, exitCode: 1 }],
]);
const result = await executePrompt("codex", "fail", {
spawn: mockSpawn(scenarios),
existsSync: () => true,
});
assert.strictEqual(result.exitCode, 1);
assert.ok(result.stderr.includes("ReasoningSummaryDelta"));
assert.ok(result.stderr.includes("something actually went wrong"));
});
it("provides rawStderr in debug info when noise is filtered", async () => {
const noisyStderr = '2026-05-20T18:33:01.969310Z ERROR codex_core::util: ReasoningSummaryDelta without active item\n';
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo hello", { stdout: "ok", stderr: noisyStderr, exitCode: 0 }],
]);
const debugInfos: any[] = [];
const result = await executePrompt("codex", "hello", {
spawn: mockSpawn(scenarios),
existsSync: () => true,
debug: true,
onDebug: (info) => debugInfos.push(info),
});
assert.strictEqual(result.exitCode, 0);
assert.strictEqual(result.stderr, "");
assert.strictEqual(debugInfos[0].rawStderr, noisyStderr);
});
it("does not set rawStderr when no noise filtering occurred", async () => {
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo hello", { stdout: "ok", stderr: "", exitCode: 0 }],
]);
const debugInfos: any[] = [];
await executePrompt("codex", "hello", {
spawn: mockSpawn(scenarios),
existsSync: () => true,
debug: true,
onDebug: (info) => debugInfos.push(info),
});
assert.strictEqual(debugInfos[0].rawStderr, undefined);
});
it("does not filter stderr for non-codex clients", async () => {
const noisyStderr = '2026-05-20T18:33:01.969310Z ERROR codex_core::util: ReasoningSummaryDelta without active item\n';
const scenarios = new Map<string, MockScenario>([
["claude -p hello --dangerously-skip-permissions", { stdout: "ok", stderr: noisyStderr, exitCode: 0 }],
]);
const result = await executePrompt("claude", "hello", {
spawn: mockSpawn(scenarios),
existsSync: () => true,
});
assert.strictEqual(result.exitCode, 0);
assert.strictEqual(result.stderr, noisyStderr);
});
});
+738
View File
@@ -0,0 +1,738 @@
import { describe, it } from "node:test";
import assert from "node:assert";
import { EventEmitter } from "node:events";
import { Readable } from "node:stream";
import {
startJob,
getJob,
getJobResult,
cancelJob,
listJobs,
cleanupJobs,
} from "../src/jobs.js";
import { JobNotFoundError, JobResultUnavailableError } from "../src/types.js";
import type { ClientName, JobRecord, JobStatus } from "../src/types.js";
interface MockScenario {
stdout?: string;
stderr?: string;
exitCode?: number;
error?: NodeJS.ErrnoException;
hang?: boolean;
}
function createMockChildProcess(scenario: MockScenario): any {
const child = new EventEmitter() as any;
child.pid = 12345;
child.stdout = Readable.from(
scenario.stdout !== undefined ? [scenario.stdout] : []
);
child.stderr = Readable.from(
scenario.stderr !== undefined ? [scenario.stderr] : []
);
child.killed = false;
child.kill = (signal: string = "SIGTERM") => {
child.killed = true;
process.nextTick(() => {
child.emit("exit", null, signal);
child.emit("close", null, signal);
});
return true;
};
child.unref = () => {};
let stdoutEnded = scenario.stdout === undefined;
let stderrEnded = scenario.stderr === undefined;
child.stdout.on("end", () => {
stdoutEnded = true;
maybeClose();
});
child.stderr.on("end", () => {
stderrEnded = true;
maybeClose();
});
function maybeClose() {
if (stdoutEnded && stderrEnded && !scenario.hang && !scenario.error) {
child.emit("exit", scenario.exitCode ?? 0, null);
child.emit("close", scenario.exitCode ?? 0, null);
}
}
process.nextTick(() => {
if (scenario.error) {
child.emit("error", scenario.error);
}
});
return child;
}
function createMockFs() {
const files = new Map<string, string>();
const dirs = new Set<string>();
const fs = {
mkdirSync: (p: string, _opts?: any) => {
dirs.add(p);
},
writeFileSync: (p: string, data: string) => {
files.set(p, data);
},
readFileSync: (p: string, _opts?: any): string => {
if (!files.has(p)) {
const err = Object.assign(new Error(`ENOENT: ${p}`), { code: "ENOENT" });
throw err;
}
return files.get(p)!;
},
readdirSync: (p: string): string[] => {
const result: string[] = [];
for (const f of files.keys()) {
const dir = f.substring(0, f.lastIndexOf("/"));
if (dir === p) {
result.push(f.substring(f.lastIndexOf("/") + 1));
}
}
return result;
},
existsSync: (p: string): boolean => {
return files.has(p) || dirs.has(p);
},
statSync: (p: string): { mtimeMs: number; isFile: () => boolean } => {
if (!files.has(p)) {
const err = Object.assign(new Error(`ENOENT: ${p}`), { code: "ENOENT" });
throw err;
}
const data = files.get(p)!;
// Use startedAt from JSON for deterministic age tests
try {
const record = JSON.parse(data) as JobRecord;
return { mtimeMs: new Date(record.startedAt).getTime(), isFile: () => true };
} catch {
return { mtimeMs: Date.now(), isFile: () => true };
}
},
unlinkSync: (p: string) => {
files.delete(p);
},
__files: files,
__dirs: dirs,
};
return fs;
}
function mockSpawn(scenarios: Map<string, MockScenario>): any {
return (cmd: string, args: string[], _opts: any): any => {
const key = [cmd, ...args].join(" ");
const scenario = scenarios.get(key);
if (!scenario) {
return createMockChildProcess({
error: Object.assign(new Error("spawn ENOENT"), { code: "ENOENT" }),
});
}
return createMockChildProcess(scenario);
};
}
/**
* Creates a mock spawnWatcher that simulates the job-watcher.ts behavior:
* spawns the mock child, captures output, and writes the final job record.
*/
function createMockWatcher(
scenarios: Map<string, MockScenario>,
fs: ReturnType<typeof createMockFs>,
opts?: { watcherTimeoutMs?: number }
): (jobFilePath: string, command: string, args: string[]) => { pid: number; unref: () => void } {
return (jobFilePath: string, command: string, clientArgs: string[]) => {
const key = [command, ...clientArgs].join(" ");
const scenario = scenarios.get(key) ?? { error: Object.assign(new Error("spawn ENOENT"), { code: "ENOENT" }) };
const child = createMockChildProcess(scenario);
const watcherPid = 99999;
let stdout = "";
let stderr = "";
child.stdout?.on("data", (chunk: Buffer | string) => { stdout += chunk.toString(); });
child.stderr?.on("data", (chunk: Buffer | string) => { stderr += chunk.toString(); });
child.on("close", (code: number | null) => {
// Read current record, update with results
let record: JobRecord;
try {
record = JSON.parse(fs.readFileSync(jobFilePath)) as JobRecord;
} catch {
return;
}
const durationMs = Date.now() - new Date(record.startedAt).getTime();
const status = code === 0 || code === null ? "completed" : "failed";
record.status = status;
record.stdout = stdout;
record.stderr = stderr;
record.completedAt = new Date().toISOString();
record.result = { stdout, stderr, exitCode: code ?? 0, client: record.client, durationMs };
if (scenario.error) {
record.status = "failed";
record.error = scenario.error.message;
}
fs.writeFileSync(jobFilePath, JSON.stringify(record));
});
child.on("error", (err: NodeJS.ErrnoException) => {
let record: JobRecord;
try {
record = JSON.parse(fs.readFileSync(jobFilePath)) as JobRecord;
} catch {
return;
}
record.status = "failed";
record.error = err.message;
record.completedAt = new Date().toISOString();
fs.writeFileSync(jobFilePath, JSON.stringify(record));
});
// For hang scenarios, simulate a timeout by writing timed_out after a delay
if (scenario.hang) {
const watcherTimeout = opts?.watcherTimeoutMs ?? 30;
setTimeout(() => {
try {
const existing = JSON.parse(fs.readFileSync(jobFilePath)) as JobRecord;
// Don't overwrite if already cancelled/completed
if (existing.status !== "running") return;
existing.status = "timed_out";
existing.completedAt = new Date().toISOString();
existing.result = { stdout, stderr, exitCode: -1, client: existing.client, durationMs: watcherTimeout };
fs.writeFileSync(jobFilePath, JSON.stringify(existing));
} catch { /* ignore */ }
}, watcherTimeout);
}
// Simulate watcher updating the PID in the job file
try {
const record = JSON.parse(fs.readFileSync(jobFilePath)) as JobRecord;
record.pid = child.pid;
fs.writeFileSync(jobFilePath, JSON.stringify(record));
} catch { /* ignore */ }
return { pid: watcherPid, unref: () => {} };
};
}
function createJobTestHelper(scenarios: Map<string, MockScenario>, jobDir: string) {
const fs = createMockFs();
const spawnWatcher = createMockWatcher(scenarios, fs);
const spawn = mockSpawn(scenarios);
return { fs, spawn, spawnWatcher, jobDir };
}
function readJobRecord(fs: ReturnType<typeof createMockFs>, path: string): JobRecord {
return JSON.parse(fs.readFileSync(path)) as JobRecord;
}
function delay(ms: number): Promise<void> {
return new Promise((r) => setTimeout(r, ms));
}
describe("startJob", () => {
it("spawns a detached child process and returns a running Job", async () => {
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo hello world", { stdout: "ok\n", exitCode: 0 }],
]);
const fs = createMockFs();
const jobDir = "/tmp/jobs";
const job = await startJob("codex", "hello world", {
jobDir,
spawn: mockSpawn(scenarios),
spawnWatcher: createMockWatcher(scenarios, fs),
fs,
});
assert.strictEqual(job.client, "codex");
assert.strictEqual(job.prompt, "hello world");
assert.strictEqual(job.status, "running");
assert.strictEqual(typeof job.id, "string");
assert.ok(job.id.length > 0);
assert.strictEqual(typeof job.pid, "number");
assert.ok(typeof job.startedAt === "string");
// Wait for child to finish
await delay(50);
const record = readJobRecord(fs, `${jobDir}/${job.id}.json`);
assert.strictEqual(record.status, "completed");
assert.strictEqual(record.stdout, "ok\n");
assert.strictEqual(record.result?.exitCode, 0);
assert.ok(typeof record.completedAt === "string");
});
it("generates unique ids for concurrent jobs", async () => {
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo a", { stdout: "a", exitCode: 0 }],
["codex exec --yolo b", { stdout: "b", exitCode: 0 }],
]);
const fs = createMockFs();
const jobDir = "/tmp/jobs";
const job1 = await startJob("codex", "a", {
jobDir,
spawn: mockSpawn(scenarios),
spawnWatcher: createMockWatcher(scenarios, fs),
fs,
});
const job2 = await startJob("codex", "b", {
jobDir,
spawn: mockSpawn(scenarios),
spawnWatcher: createMockWatcher(scenarios, fs),
fs,
});
assert.notStrictEqual(job1.id, job2.id);
});
it("sets status to timed_out when timeout is exceeded", async () => {
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo slow", { hang: true }],
]);
const fs = createMockFs();
const jobDir = "/tmp/jobs";
const job = await startJob("codex", "slow", {
jobDir,
spawn: mockSpawn(scenarios),
spawnWatcher: createMockWatcher(scenarios, fs),
fs,
timeoutMs: 20,
});
// Wait for timeout + processing
await delay(100);
const record = readJobRecord(fs, `${jobDir}/${job.id}.json`);
assert.strictEqual(record.status, "timed_out");
});
it("sets status to failed on non-zero exit code", async () => {
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo fail", { stdout: "", stderr: "err", exitCode: 1 }],
]);
const fs = createMockFs();
const jobDir = "/tmp/jobs";
const job = await startJob("codex", "fail", {
jobDir,
spawn: mockSpawn(scenarios),
spawnWatcher: createMockWatcher(scenarios, fs),
fs,
});
await delay(50);
const record = readJobRecord(fs, `${jobDir}/${job.id}.json`);
assert.strictEqual(record.status, "failed");
assert.strictEqual(record.result?.exitCode, 1);
assert.strictEqual(record.stderr, "err");
});
it("sets status to failed on spawn ENOENT", async () => {
const scenarios = new Map<string, MockScenario>();
const fs = createMockFs();
const jobDir = "/tmp/jobs";
const job = await startJob("codex", "hello", {
jobDir,
spawn: mockSpawn(scenarios),
spawnWatcher: createMockWatcher(scenarios, fs),
fs,
});
await delay(50);
const record = readJobRecord(fs, `${jobDir}/${job.id}.json`);
assert.strictEqual(record.status, "failed");
assert.ok(record.error);
});
});
describe("getJob", () => {
it("returns the current Job state from disk", async () => {
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo hello", { hang: true }],
]);
const fs = createMockFs();
const jobDir = "/tmp/jobs";
const job = await startJob("codex", "hello", {
jobDir,
spawn: mockSpawn(scenarios),
spawnWatcher: createMockWatcher(scenarios, fs, { watcherTimeoutMs: 5000 }),
fs,
});
const fetched = getJob(job.id, { jobDir, fs });
assert.strictEqual(fetched.id, job.id);
assert.strictEqual(fetched.status, "running");
await delay(50);
// Job should still be running since watcher timeout is 5s
const after = getJob(job.id, { jobDir, fs });
assert.strictEqual(after.status, "running");
});
it("throws JobNotFoundError for nonexistent job", () => {
const fs = createMockFs();
assert.throws(
() => getJob("missing", { jobDir: "/tmp/jobs", fs }),
(err: unknown) => err instanceof JobNotFoundError
);
});
});
describe("getJobResult", () => {
it("returns ExecResult when job is completed", async () => {
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo hello", { stdout: "ok", exitCode: 0 }],
]);
const fs = createMockFs();
const jobDir = "/tmp/jobs";
const job = await startJob("codex", "hello", {
jobDir,
spawn: mockSpawn(scenarios),
spawnWatcher: createMockWatcher(scenarios, fs),
fs,
});
await delay(50);
const result = getJobResult(job.id, { jobDir, fs });
assert.strictEqual(result.stdout, "ok");
assert.strictEqual(result.exitCode, 0);
});
it("throws JobResultUnavailableError when job is still running", async () => {
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo hello", { hang: true }],
]);
const fs = createMockFs();
const jobDir = "/tmp/jobs";
const job = await startJob("codex", "hello", {
jobDir,
spawn: mockSpawn(scenarios),
spawnWatcher: createMockWatcher(scenarios, fs),
fs,
timeoutMs: 50,
});
assert.throws(
() => getJobResult(job.id, { jobDir, fs }),
(err: unknown) => err instanceof JobResultUnavailableError
);
});
it("throws JobResultUnavailableError when job failed", async () => {
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo fail", { stdout: "", stderr: "err", exitCode: 1 }],
]);
const fs = createMockFs();
const jobDir = "/tmp/jobs";
const job = await startJob("codex", "fail", {
jobDir,
spawn: mockSpawn(scenarios),
spawnWatcher: createMockWatcher(scenarios, fs),
fs,
});
await delay(50);
assert.throws(
() => getJobResult(job.id, { jobDir, fs }),
(err: unknown) => err instanceof JobResultUnavailableError
);
});
it("throws JobNotFoundError for nonexistent job", () => {
const fs = createMockFs();
assert.throws(
() => getJobResult("missing", { jobDir: "/tmp/jobs", fs }),
(err: unknown) => err instanceof JobNotFoundError
);
});
});
describe("cancelJob", () => {
it("sends SIGTERM and updates status to cancelled", async () => {
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo hello", { hang: true }],
]);
const fs = createMockFs();
const jobDir = "/tmp/jobs";
const job = await startJob("codex", "hello", {
jobDir,
spawn: mockSpawn(scenarios),
spawnWatcher: createMockWatcher(scenarios, fs, { watcherTimeoutMs: 5000 }),
fs,
});
cancelJob(job.id, { jobDir, fs });
await delay(50);
const record = readJobRecord(fs, `${jobDir}/${job.id}.json`);
assert.strictEqual(record.status, "cancelled");
});
it("throws JobNotFoundError for nonexistent job", () => {
const fs = createMockFs();
assert.throws(
() => cancelJob("missing", { jobDir: "/tmp/jobs", fs }),
(err: unknown) => err instanceof JobNotFoundError
);
});
it("is a no-op when job is not running", () => {
const fs = createMockFs();
const jobDir = "/tmp/jobs";
const now = new Date().toISOString();
const completedRecord: JobRecord = {
id: "job-completed",
client: "codex",
prompt: "done",
status: "completed",
startedAt: now,
completedAt: now,
stdout: "ok",
stderr: "",
result: {
stdout: "ok",
stderr: "",
exitCode: 0,
client: "codex",
durationMs: 100,
},
};
fs.mkdirSync(jobDir, { recursive: true });
fs.writeFileSync(`${jobDir}/job-completed.json`, JSON.stringify(completedRecord));
cancelJob("job-completed", { jobDir, fs });
const record = readJobRecord(fs, `${jobDir}/job-completed.json`);
assert.strictEqual(record.status, "completed");
});
});
describe("listJobs", () => {
it("returns all jobs sorted by startedAt desc", async () => {
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo a", { stdout: "a", exitCode: 0 }],
["codex exec --yolo b", { stdout: "b", exitCode: 0 }],
]);
const fs = createMockFs();
const jobDir = "/tmp/jobs";
const job1 = await startJob("codex", "a", {
jobDir,
spawn: mockSpawn(scenarios),
spawnWatcher: createMockWatcher(scenarios, fs),
fs,
});
await delay(20);
const job2 = await startJob("codex", "b", {
jobDir,
spawn: mockSpawn(scenarios),
spawnWatcher: createMockWatcher(scenarios, fs),
fs,
});
await delay(50);
const jobs = listJobs({ jobDir, fs });
assert.strictEqual(jobs.length, 2);
assert.strictEqual(jobs[0].id, job2.id);
assert.strictEqual(jobs[1].id, job1.id);
});
it("returns empty array when jobDir does not exist", () => {
const fs = createMockFs();
const jobs = listJobs({ jobDir: "/tmp/jobs", fs });
assert.deepStrictEqual(jobs, []);
});
it("ignores corrupt or unreadable job files", () => {
const fs = createMockFs();
const jobDir = "/tmp/jobs";
const now = new Date().toISOString();
const validRecord: JobRecord = {
id: "job-valid",
client: "codex",
prompt: "ok",
status: "completed",
startedAt: now,
completedAt: now,
stdout: "ok",
stderr: "",
result: {
stdout: "ok",
stderr: "",
exitCode: 0,
client: "codex",
durationMs: 100,
},
};
fs.mkdirSync(jobDir, { recursive: true });
fs.writeFileSync(`${jobDir}/job-valid.json`, JSON.stringify(validRecord));
fs.writeFileSync(`${jobDir}/job-corrupt.json`, "not-json");
const jobs = listJobs({ jobDir, fs });
assert.strictEqual(jobs.length, 1);
assert.strictEqual(jobs[0].id, "job-valid");
});
it("filters jobs by status when filter is provided", () => {
const fs = createMockFs();
const jobDir = "/tmp/jobs";
const now = new Date().toISOString();
const runningRecord: JobRecord = {
id: "job-running",
client: "codex",
prompt: "run",
status: "running",
startedAt: now,
stdout: "",
stderr: "",
};
const completedRecord: JobRecord = {
id: "job-completed",
client: "claude",
prompt: "done",
status: "completed",
startedAt: now,
completedAt: now,
stdout: "ok",
stderr: "",
result: {
stdout: "ok",
stderr: "",
exitCode: 0,
client: "claude",
durationMs: 100,
},
};
const failedRecord: JobRecord = {
id: "job-failed",
client: "opencode",
prompt: "fail",
status: "failed",
startedAt: now,
completedAt: now,
stdout: "",
stderr: "err",
result: {
stdout: "",
stderr: "err",
exitCode: 1,
client: "opencode",
durationMs: 100,
},
};
fs.mkdirSync(jobDir, { recursive: true });
fs.writeFileSync(`${jobDir}/job-running.json`, JSON.stringify(runningRecord));
fs.writeFileSync(`${jobDir}/job-completed.json`, JSON.stringify(completedRecord));
fs.writeFileSync(`${jobDir}/job-failed.json`, JSON.stringify(failedRecord));
const all = listJobs({ jobDir, fs });
assert.strictEqual(all.length, 3);
const running = listJobs({ jobDir, fs, filter: "running" });
assert.strictEqual(running.length, 1);
assert.strictEqual(running[0].id, "job-running");
const completed = listJobs({ jobDir, fs, filter: "completed" });
assert.strictEqual(completed.length, 1);
assert.strictEqual(completed[0].id, "job-completed");
const failed = listJobs({ jobDir, fs, filter: "failed" });
assert.strictEqual(failed.length, 1);
assert.strictEqual(failed[0].id, "job-failed");
});
});
describe("cleanupJobs", () => {
it("deletes job files older than maxAgeMs", async () => {
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo old", { stdout: "old", exitCode: 0 }],
["codex exec --yolo new", { stdout: "new", exitCode: 0 }],
]);
const fs = createMockFs();
const jobDir = "/tmp/jobs";
// Create an old job by manipulating startedAt after creation
const oldJob = await startJob("codex", "old", {
jobDir,
spawn: mockSpawn(scenarios),
spawnWatcher: createMockWatcher(scenarios, fs),
fs,
});
await delay(50);
const newJob = await startJob("codex", "new", {
jobDir,
spawn: mockSpawn(scenarios),
spawnWatcher: createMockWatcher(scenarios, fs),
fs,
});
await delay(50);
// Make old job appear 25h old by patching its record
const oldPath = `${jobDir}/${oldJob.id}.json`;
const oldRecord = readJobRecord(fs, oldPath);
oldRecord.startedAt = new Date(Date.now() - 25 * 60 * 60 * 1000).toISOString();
fs.writeFileSync(oldPath, JSON.stringify(oldRecord));
cleanupJobs({ jobDir, fs, maxAgeMs: 24 * 60 * 60 * 1000 });
assert.strictEqual(fs.existsSync(oldPath), false);
assert.strictEqual(fs.existsSync(`${jobDir}/${newJob.id}.json`), true);
});
it("uses default maxAge of 24 hours", async () => {
const scenarios = new Map<string, MockScenario>([
["codex exec --yolo old", { stdout: "old", exitCode: 0 }],
]);
const fs = createMockFs();
const jobDir = "/tmp/jobs";
const job = await startJob("codex", "old", {
jobDir,
spawn: mockSpawn(scenarios),
spawnWatcher: createMockWatcher(scenarios, fs),
fs,
});
await delay(50);
const path = `${jobDir}/${job.id}.json`;
const record = readJobRecord(fs, path);
record.startedAt = new Date(Date.now() - 25 * 60 * 60 * 1000).toISOString();
fs.writeFileSync(path, JSON.stringify(record));
cleanupJobs({ jobDir, fs });
assert.strictEqual(fs.existsSync(path), false);
});
it("is a no-op when jobDir does not exist", () => {
const fs = createMockFs();
// Should not throw
cleanupJobs({ jobDir: "/tmp/jobs", fs });
assert.strictEqual(fs.existsSync("/tmp/jobs"), false);
});
});
+17
View File
@@ -0,0 +1,17 @@
{
"compilerOptions": {
"target": "ES2022",
"module": "ESNext",
"moduleResolution": "bundler",
"esModuleInterop": true,
"allowSyntheticDefaultImports": true,
"strict": true,
"skipLibCheck": true,
"resolveJsonModule": true,
"types": ["node"],
"outDir": "./dist",
"rootDir": "."
},
"include": ["src/**/*.ts", "tests/**/*.ts"],
"exclude": ["node_modules", "dist"]
}