feat(S-402): Update docs/ai-cli-dispatch.md and docs/architecture.md

2026-05-19 22:45:38 -05:00
parent bd88df7dd2
commit 601f7cce89
2 changed files with 340 additions and 54 deletions
@@ -9,8 +9,10 @@ Dispatch AI CLI coding tasks to available clients (Codex, Claude Code, OpenCode)
 - dispatch a prompt to a specific client by name
 - auto-resolve the best client from prompt keywords
 - forward arguments natively to each client
+- run tasks asynchronously as background jobs with lifecycle management
+- run tasks synchronously when blocking until completion is desired

-The tool is a lightweight sync-only dispatcher. It does not implement streaming, chat sessions, or ACP orchestration. For ACP-based harnesses, see `docs/openclaw-acp-orchestration.md`.
+The tool supports both async (default) and sync execution modes. Async jobs run as detached background processes and are tracked on disk. For ACP-based harnesses, see `docs/openclaw-acp-orchestration.md`.

 ## Setup

@@ -27,8 +29,14 @@ The dispatcher itself requires only Node.js 20+ and `npm`. The actual AI CLI cli

 ```bash
 ai-cli-dispatch list [--json|--text]
-ai-cli-dispatch run --client <client> --prompt <prompt> [--json|--text]
-ai-cli-dispatch dispatch <prompt> [--client <client>] [--json|--text]
+ai-cli-dispatch run --client <client> --prompt <prompt> [--sync] [--timeout <ms>] [--debug] [--json|--text]
+ai-cli-dispatch dispatch <prompt> [--client <client>] [--sync] [--timeout <ms>] [--debug] [--json|--text]
+ai-cli-dispatch start --client <client> --prompt <prompt> [--timeout <ms>] [--debug] [--json|--text]
+ai-cli-dispatch status <job-id> [--json|--text]
+ai-cli-dispatch results <job-id> [--json|--text]
+ai-cli-dispatch cancel <job-id> [--json|--text]
+ai-cli-dispatch list-jobs [--status running|completed|failed] [--json|--text]
+ai-cli-dispatch cleanup-jobs [--max-age <number>[h|m|s|d]] [--json|--text]
 ai-cli-dispatch --help
 ```

@@ -71,12 +79,17 @@ ai-cli-dispatch list --text

 ### `run`

-Execute a prompt directly through a named client.
+Execute a prompt directly through a named client. By default, this starts a background job and returns immediately.

 ```bash
+# Async (default) — returns a job ID
 ai-cli-dispatch run --client codex --prompt "refactor this function"
-ai-cli-dispatch run --client claude --prompt "add tests for auth middleware"
-ai-cli-dispatch run --client opencode --prompt "migrate to ESM"
+
+# Sync — blocks until the client finishes
+ai-cli-dispatch run --client claude --prompt "add tests for auth middleware" --sync
+
+# With custom timeout and debug diagnostics
+ai-cli-dispatch run --client opencode --prompt "migrate to ESM" --timeout 600000 --debug
 ```

 The prompt is forwarded with each client’s native argument shape:
@@ -89,7 +102,7 @@ The prompt is forwarded with each client’s native argument shape:

 ### `dispatch`

-Auto-resolve the client from prompt keywords, then execute.
+Auto-resolve the client from prompt keywords, then execute. Defaults to async; use `--sync` to block.

 ```bash
 ai-cli-dispatch dispatch "use claude to write tests"
@@ -112,6 +125,106 @@ Override auto-resolution explicitly:
 ai-cli-dispatch dispatch "fix the bug" --client claude
 ```

+### `start`
+
+Explicitly start a background job (same as `run` without `--sync`). Useful when you want the async behavior unambiguously.
+
+```bash
+ai-cli-dispatch start --client codex --prompt "refactor this function"
+```
+
+### `status`
+
+Check the status of a background job.
+
+```bash
+ai-cli-dispatch status <job-id>
+```
+
+JSON output:
+
+```json
+{
+  "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
+  "client": "codex",
+  "prompt": "refactor this function",
+  "status": "running",
+  "startedAt": "2026-05-19T12:34:56.789Z",
+  "pid": 12345
+}
+```
+
+Statuses: `running`, `completed`, `failed`, `timed_out`, `cancelled`.
+
+### `results`
+
+Retrieve the execution result of a completed job.
+
+```bash
+ai-cli-dispatch results <job-id>
+```
+
+JSON output:
+
+```json
+{
+  "stdout": "...",
+  "stderr": "...",
+  "exitCode": 0,
+  "client": "codex",
+  "durationMs": 45231
+}
+```
+
+Requires status `completed`. For `failed` or `timed_out` jobs, use `status` to see the captured error.
+
+### `cancel`
+
+Cancel a running job.
+
+```bash
+ai-cli-dispatch cancel <job-id>
+```
+
+### `list-jobs`
+
+List all tracked jobs, newest first.
+
+```bash
+ai-cli-dispatch list-jobs --json
+ai-cli-dispatch list-jobs --status running --json
+```
+
+### `cleanup-jobs`
+
+Remove job files older than a threshold. Default unit is hours.
+
+```bash
+ai-cli-dispatch cleanup-jobs --max-age 24h
+ai-cli-dispatch cleanup-jobs --max-age 30m
+```
+
+## Async vs Sync Mode
+
+By default, `run` and `dispatch` are **async**: they start a detached background process, persist a job record to disk, and return a job ID immediately. This is ideal for:
+
+- Fire-and-forget tasks that may run for minutes
+- Long-running codegen or migration tasks
+- Scenarios where the caller should not block
+
+Use `--sync` when you need:
+
+- The complete output before the next step
+- Synchronous composition in shell pipelines or scripts
+- Immediate error propagation to the calling process
+
+| Aspect | Async (default) | Sync (`--sync`) |
+|---|---|---|
+| Return value | Job ID + status | Full stdout/stderr + exit code |
+| Process model | Detached child, parent exits immediately | Attached child, parent waits |
+| Persistence | Job file written to disk | No job file |
+| Timeout | Enforced via `child.kill()` after `--timeout` | Enforced via `child.kill()` after `--timeout` |
+
 ## Client Discovery

 Discovery searches `PATH` in this order for each client name:
@@ -138,7 +251,8 @@ Example:
    "codex": "/usr/local/bin/codex",
    "claude": "/opt/homebrew/bin/claude"
  },
-  "defaultClient": "claude"
+  "defaultClient": "claude",
+  "timeout": 300000
 }
 ```

@@ -162,17 +276,54 @@ Supported env vars:

 Default output is JSON. Use `--text` to stream raw `stdout`/`stderr` directly.

-JSON success shape (`run` and `dispatch`):
+### Sync JSON success shape (`run --sync`, `dispatch --sync`)

 ```json
 {
  "stdout": "...",
  "stderr": "...",
-  "exitCode": 0
+  "exitCode": 0,
+  "client": "codex",
+  "durationMs": 45231
 }
 ```

-JSON error shape:
+### Async JSON success shape (`run`, `dispatch`, `start`)
+
+```json
+{
+  "jobId": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
+  "client": "codex",
+  "status": "running"
+}
+```
+
+### Job status shape (`status`)
+
+```json
+{
+  "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
+  "client": "codex",
+  "prompt": "refactor this function",
+  "status": "running",
+  "startedAt": "2026-05-19T12:34:56.789Z",
+  "pid": 12345
+}
+```
+
+### Job result shape (`results`)
+
+```json
+{
+  "stdout": "...",
+  "stderr": "...",
+  "exitCode": 0,
+  "client": "codex",
+  "durationMs": 45231
+}
+```
+
+### JSON error shape

 ```json
 {
@@ -185,7 +336,7 @@ Exit codes:
 | Code | Meaning |
 |---|---|
 | `0` | Success |
-| `1` | Missing/unknown command, missing argument, unknown client, resolution failure, or execution error |
+| `1` | Missing/unknown command, missing argument, unknown client, resolution failure, execution error, or job lifecycle error |

 ## Error Handling Guidance

@@ -205,11 +356,11 @@ Meaning: the prompt string was empty or whitespace-only.

 Action: supply a non-empty `--prompt` or positional prompt argument.

-### `Execution timed out after 300000ms`
+### `Execution timed out after 600000ms`

-Meaning: the client subprocess did not finish within the default 5-minute timeout.
+Meaning: the client subprocess did not finish within the timeout.

-Action: the client may be waiting for interactive input or the task is too large. Break the prompt into smaller pieces, or run the client directly to diagnose.
+Action: the client may be waiting for interactive input or the task is too large. Break the prompt into smaller pieces, increase `--timeout`, or run the client directly to diagnose. Async jobs that time out are recorded with status `timed_out`.

 ### `Could not resolve client from prompt`

@@ -217,6 +368,49 @@ Meaning: `dispatch` found no matching keyword and no `defaultClient` is configur

 Action: include a client name in the prompt (e.g., `"use claude to ..."`) or set `defaultClient` in config.

+### `Job "<job-id>" not found`
+
+Meaning: the requested job ID does not exist in the job store.
+
+Action: verify the job ID. Job files are stored under `~/.openclaw/ai-cli-dispatch/jobs/`. If the directory was cleaned or the host restarted, old jobs may have been removed.
+
+### `Job "<job-id>" result is not available (status: <status>)`
+
+Meaning: `results` was called on a job that has not finished (`running`) or finished unsuccessfully (`failed`, `timed_out`, `cancelled`).
+
+Action: poll `status` until the job reaches `completed`, or inspect `status` output for the error field.
+
+## Job Lifecycle Workflows
+
+### Fire-and-forget
+
+```bash
+JOB=$(ai-cli-dispatch run --client codex --prompt "refactor auth" --json | jq -r '.jobId')
+# caller continues immediately
+```
+
+### Poll until completion
+
+```bash
+JOB=$(ai-cli-dispatch start --client claude --prompt "write tests" --json | jq -r '.jobId')
+while [ "$(ai-cli-dispatch status "$JOB" --json | jq -r '.status')" = "running" ]; do
+  sleep 5
+done
+ai-cli-dispatch results "$JOB" --json
+```
+
+### Sync one-shot
+
+```bash
+ai-cli-dispatch run --client opencode --prompt "fix lint" --sync --text
+```
+
+### Batch cleanup
+
+```bash
+ai-cli-dispatch cleanup-jobs --max-age 24h
+```
+
 ## Common Flows

 ### Check what is installed
@@ -225,12 +419,18 @@ Action: include a client name in the prompt (e.g., `"use claude to ..."`) or set
 ai-cli-dispatch list --json
 ```

-### Run a quick task through a specific client
+### Run a quick task through a specific client (async)

 ```bash
 ai-cli-dispatch run --client codex --prompt "fix lint errors in src/app.ts"
 ```

+### Run a quick task synchronously
+
+```bash
+ai-cli-dispatch run --client codex --prompt "fix lint errors in src/app.ts" --sync
+```
+
 ### Let the tool pick the client from the prompt

 ```bash
@@ -245,14 +445,16 @@ ai-cli-dispatch dispatch "review this PR" --client claude

 ## Coexistence with ACP

-`ai-cli-dispatch` is a direct subprocess dispatcher. It runs the client binary synchronously and returns its output. It is not an ACP agent and does not participate in ACP orchestration.
+`ai-cli-dispatch` is a direct subprocess dispatcher. It is not an ACP agent and does not participate in ACP orchestration.

- Use `ai-cli-dispatch` when you need a quick, local, one-shot CLI execution.
+- Use `ai-cli-dispatch` when you need a quick, local, one-shot CLI execution or a background job.
 - Use ACP (`docs/openclaw-acp-orchestration.md`) when you need session-bound coding harnesses with thread context, multi-turn review, or orchestrator-managed verification gates.

 ## Implementation Notes

 - The dispatcher is TypeScript/Node.js with a single external dependency (`minimist`).
 - Client arguments are hardcoded per tool to match each client’s stable CLI contract.
- The default timeout is 5 minutes (`300_000` ms).
+- The default timeout is 10 minutes (`600_000` ms); override with `--timeout` or config.
 - On Windows, discovery uses `where` instead of `which` and `.exe` extensions are assumed.
+- Async jobs run as detached processes with `stdio: ["ignore", "pipe", "pipe"]` so the dispatcher can exit without waiting.
+- Job files are written atomically to `~/.openclaw/ai-cli-dispatch/jobs/<jobId>.json`.