merge M4 into implement/2026-05-19-ai-cli-dispatch-fixes

2026-05-19 22:49:05 -05:00
parent bd88df7dd2 7b886a7b33
commit edb6611b74
3 changed files with 470 additions and 63 deletions
@@ -9,8 +9,10 @@ Dispatch AI CLI coding tasks to available clients (Codex, Claude Code, OpenCode)
 - dispatch a prompt to a specific client by name
 - auto-resolve the best client from prompt keywords
 - forward arguments natively to each client
+- run tasks asynchronously as background jobs with lifecycle management
+- run tasks synchronously when blocking until completion is desired

-The tool is a lightweight sync-only dispatcher. It does not implement streaming, chat sessions, or ACP orchestration. For ACP-based harnesses, see `docs/openclaw-acp-orchestration.md`.
+The tool supports both async (default) and sync execution modes. Async jobs run as detached background processes and are tracked on disk. For ACP-based harnesses, see `docs/openclaw-acp-orchestration.md`.

 ## Setup

@@ -27,8 +29,14 @@ The dispatcher itself requires only Node.js 20+ and `npm`. The actual AI CLI cli

 ```bash
 ai-cli-dispatch list [--json|--text]
-ai-cli-dispatch run --client <client> --prompt <prompt> [--json|--text]
-ai-cli-dispatch dispatch <prompt> [--client <client>] [--json|--text]
+ai-cli-dispatch run --client <client> --prompt <prompt> [--sync] [--timeout <ms>] [--debug] [--json|--text]
+ai-cli-dispatch dispatch <prompt> [--client <client>] [--sync] [--timeout <ms>] [--debug] [--json|--text]
+ai-cli-dispatch start --client <client> --prompt <prompt> [--timeout <ms>] [--debug] [--json|--text]
+ai-cli-dispatch status <job-id> [--json|--text]
+ai-cli-dispatch results <job-id> [--json|--text]
+ai-cli-dispatch cancel <job-id> [--json|--text]
+ai-cli-dispatch list-jobs [--status running|completed|failed] [--json|--text]
+ai-cli-dispatch cleanup-jobs [--max-age <number>[h|m|s|d]] [--json|--text]
 ai-cli-dispatch --help
 ```

@@ -71,25 +79,30 @@ ai-cli-dispatch list --text

 ### `run`

-Execute a prompt directly through a named client.
+Execute a prompt directly through a named client. By default, this starts a background job and returns immediately.

 ```bash
+# Async (default) — returns a job ID
 ai-cli-dispatch run --client codex --prompt "refactor this function"
-ai-cli-dispatch run --client claude --prompt "add tests for auth middleware"
-ai-cli-dispatch run --client opencode --prompt "migrate to ESM"
+
+# Sync — blocks until the client finishes
+ai-cli-dispatch run --client claude --prompt "add tests for auth middleware" --sync
+
+# With custom timeout and debug diagnostics
+ai-cli-dispatch run --client opencode --prompt "migrate to ESM" --timeout 600000 --debug
 ```

 The prompt is forwarded with each client’s native argument shape:

 | Client | Arguments passed |
 |---|---|
-| `codex` | `exec "<prompt>"` |
-| `claude` | `-p "<prompt>"` |
-| `opencode` | `"<prompt>"` |
+| `codex` | `exec --yolo "<prompt>"` |
+| `claude` | `-p "<prompt>" --dangerously-skip-permissions` |
+| `opencode` | `run --dangerously-skip-permissions "<prompt>"` |

 ### `dispatch`

-Auto-resolve the client from prompt keywords, then execute.
+Auto-resolve the client from prompt keywords, then execute. Defaults to async; use `--sync` to block.

 ```bash
 ai-cli-dispatch dispatch "use claude to write tests"
@@ -112,6 +125,106 @@ Override auto-resolution explicitly:
 ai-cli-dispatch dispatch "fix the bug" --client claude
 ```

+### `start`
+
+Explicitly start a background job (same as `run` without `--sync`). Useful when you want the async behavior unambiguously.
+
+```bash
+ai-cli-dispatch start --client codex --prompt "refactor this function"
+```
+
+### `status`
+
+Check the status of a background job.
+
+```bash
+ai-cli-dispatch status <job-id>
+```
+
+JSON output:
+
+```json
+{
+  "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
+  "client": "codex",
+  "prompt": "refactor this function",
+  "status": "running",
+  "startedAt": "2026-05-19T12:34:56.789Z",
+  "pid": 12345
+}
+```
+
+Statuses: `running`, `completed`, `failed`, `timed_out`, `cancelled`.
+
+### `results`
+
+Retrieve the execution result of a completed job.
+
+```bash
+ai-cli-dispatch results <job-id>
+```
+
+JSON output:
+
+```json
+{
+  "stdout": "...",
+  "stderr": "...",
+  "exitCode": 0,
+  "client": "codex",
+  "durationMs": 45231
+}
+```
+
+Requires status `completed`. For `failed` or `timed_out` jobs, use `status` to see the captured error.
+
+### `cancel`
+
+Cancel a running job.
+
+```bash
+ai-cli-dispatch cancel <job-id>
+```
+
+### `list-jobs`
+
+List all tracked jobs, newest first.
+
+```bash
+ai-cli-dispatch list-jobs --json
+ai-cli-dispatch list-jobs --status running --json
+```
+
+### `cleanup-jobs`
+
+Remove job files older than a threshold. Default unit is hours.
+
+```bash
+ai-cli-dispatch cleanup-jobs --max-age 24h
+ai-cli-dispatch cleanup-jobs --max-age 30m
+```
+
+## Async vs Sync Mode
+
+By default, `run` and `dispatch` are **async**: they start a detached background process, persist a job record to disk, and return a job ID immediately. This is ideal for:
+
+- Fire-and-forget tasks that may run for minutes
+- Long-running codegen or migration tasks
+- Scenarios where the caller should not block
+
+Use `--sync` when you need:
+
+- The complete output before the next step
+- Synchronous composition in shell pipelines or scripts
+- Immediate error propagation to the calling process
+
+| Aspect | Async (default) | Sync (`--sync`) |
+|---|---|---|
+| Return value | Job ID + status | Full stdout/stderr + exit code |
+| Process model | Detached child, parent exits immediately | Attached child, parent waits |
+| Persistence | Job file written to disk | No job file |
+| Timeout | Enforced via `child.kill()` after `--timeout` | Enforced via `child.kill()` after `--timeout` |
+
 ## Client Discovery

 Discovery searches `PATH` in this order for each client name:
@@ -138,7 +251,8 @@ Example:
    "codex": "/usr/local/bin/codex",
    "claude": "/opt/homebrew/bin/claude"
  },
-  "defaultClient": "claude"
+  "defaultClient": "claude",
+  "timeout": 300000
 }
 ```

@@ -162,17 +276,54 @@ Supported env vars:

 Default output is JSON. Use `--text` to stream raw `stdout`/`stderr` directly.

-JSON success shape (`run` and `dispatch`):
+### Sync JSON success shape (`run --sync`, `dispatch --sync`)

 ```json
 {
  "stdout": "...",
  "stderr": "...",
-  "exitCode": 0
+  "exitCode": 0,
+  "client": "codex",
+  "durationMs": 45231
 }
 ```

-JSON error shape:
+### Async JSON success shape (`run`, `dispatch`, `start`)
+
+```json
+{
+  "jobId": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
+  "client": "codex",
+  "status": "running"
+}
+```
+
+### Job status shape (`status`)
+
+```json
+{
+  "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
+  "client": "codex",
+  "prompt": "refactor this function",
+  "status": "running",
+  "startedAt": "2026-05-19T12:34:56.789Z",
+  "pid": 12345
+}
+```
+
+### Job result shape (`results`)
+
+```json
+{
+  "stdout": "...",
+  "stderr": "...",
+  "exitCode": 0,
+  "client": "codex",
+  "durationMs": 45231
+}
+```
+
+### JSON error shape

 ```json
 {
@@ -185,7 +336,7 @@ Exit codes:
 | Code | Meaning |
 |---|---|
 | `0` | Success |
-| `1` | Missing/unknown command, missing argument, unknown client, resolution failure, or execution error |
+| `1` | Missing/unknown command, missing argument, unknown client, resolution failure, execution error, or job lifecycle error |

 ## Error Handling Guidance

@@ -205,11 +356,11 @@ Meaning: the prompt string was empty or whitespace-only.

 Action: supply a non-empty `--prompt` or positional prompt argument.

-### `Execution timed out after 300000ms`
+### `Execution timed out after 600000ms`

-Meaning: the client subprocess did not finish within the default 5-minute timeout.
+Meaning: the client subprocess did not finish within the timeout.

-Action: the client may be waiting for interactive input or the task is too large. Break the prompt into smaller pieces, or run the client directly to diagnose.
+Action: the client may be waiting for interactive input or the task is too large. Break the prompt into smaller pieces, increase `--timeout`, or run the client directly to diagnose. Async jobs that time out are recorded with status `timed_out`.

 ### `Could not resolve client from prompt`

@@ -217,6 +368,49 @@ Meaning: `dispatch` found no matching keyword and no `defaultClient` is configur

 Action: include a client name in the prompt (e.g., `"use claude to ..."`) or set `defaultClient` in config.

+### `Job "<job-id>" not found`
+
+Meaning: the requested job ID does not exist in the job store.
+
+Action: verify the job ID. Job files are stored under `~/.openclaw/ai-cli-dispatch/jobs/`. If the directory was cleaned or the host restarted, old jobs may have been removed.
+
+### `Job "<job-id>" result is not available (status: <status>)`
+
+Meaning: `results` was called on a job that has not finished (`running`) or finished unsuccessfully (`failed`, `timed_out`, `cancelled`).
+
+Action: poll `status` until the job reaches `completed`, or inspect `status` output for the error field.
+
+## Job Lifecycle Workflows
+
+### Fire-and-forget
+
+```bash
+JOB=$(ai-cli-dispatch run --client codex --prompt "refactor auth" --json | jq -r '.jobId')
+# caller continues immediately
+```
+
+### Poll until completion
+
+```bash
+JOB=$(ai-cli-dispatch start --client claude --prompt "write tests" --json | jq -r '.jobId')
+while [ "$(ai-cli-dispatch status "$JOB" --json | jq -r '.status')" = "running" ]; do
+  sleep 5
+done
+ai-cli-dispatch results "$JOB" --json
+```
+
+### Sync one-shot
+
+```bash
+ai-cli-dispatch run --client opencode --prompt "fix lint" --sync --text
+```
+
+### Batch cleanup
+
+```bash
+ai-cli-dispatch cleanup-jobs --max-age 24h
+```
+
 ## Common Flows

 ### Check what is installed
@@ -225,12 +419,18 @@ Action: include a client name in the prompt (e.g., `"use claude to ..."`) or set
 ai-cli-dispatch list --json
 ```

-### Run a quick task through a specific client
+### Run a quick task through a specific client (async)

 ```bash
 ai-cli-dispatch run --client codex --prompt "fix lint errors in src/app.ts"
 ```

+### Run a quick task synchronously
+
+```bash
+ai-cli-dispatch run --client codex --prompt "fix lint errors in src/app.ts" --sync
+```
+
 ### Let the tool pick the client from the prompt

 ```bash
@@ -245,14 +445,16 @@ ai-cli-dispatch dispatch "review this PR" --client claude

 ## Coexistence with ACP

-`ai-cli-dispatch` is a direct subprocess dispatcher. It runs the client binary synchronously and returns its output. It is not an ACP agent and does not participate in ACP orchestration.
+`ai-cli-dispatch` is a direct subprocess dispatcher. It is not an ACP agent and does not participate in ACP orchestration.

- Use `ai-cli-dispatch` when you need a quick, local, one-shot CLI execution.
+- Use `ai-cli-dispatch` when you need a quick, local, one-shot CLI execution or a background job.
 - Use ACP (`docs/openclaw-acp-orchestration.md`) when you need session-bound coding harnesses with thread context, multi-turn review, or orchestrator-managed verification gates.

 ## Implementation Notes

 - The dispatcher is TypeScript/Node.js with a single external dependency (`minimist`).
 - Client arguments are hardcoded per tool to match each client’s stable CLI contract.
- The default timeout is 5 minutes (`300_000` ms).
+- The default timeout is 10 minutes (`600_000` ms); override with `--timeout` or config.
 - On Windows, discovery uses `where` instead of `which` and `.exe` extensions are assumed.
+- Async jobs run as detached processes with `stdio: ["ignore", "pipe", "pipe"]` so the dispatcher can exit without waiting.
+- Job files are written atomically to `~/.openclaw/ai-cli-dispatch/jobs/<jobId>.json`.
@@ -6,30 +6,36 @@ This document describes the internal design of `ai-cli-dispatch`, the module bre

 ```text
 src/
-├── cli.ts        — Entry point: argument parsing, command routing, I/O formatting
-├── types.ts      — Shared types and error classes
-├── constants.ts  — Client name registry and platform helpers
-├── config.ts     — Layered configuration resolution (flags → env → file → PATH)
-├── detect.ts     — Client discovery: binary lookup and version extraction
-├── dispatch.ts   — Prompt-to-client resolution (explicit flag → keywords → default)
-└── execute.ts    — Subprocess spawning, stdout/stderr capture, timeout handling
+├── cli.ts          — Entry point: argument parsing, command routing, I/O formatting
+├── cli-helpers.ts  — Shared formatting, sync/async run handlers, error reporters
+├── types.ts        — Shared types and error classes
+├── constants.ts    — Client name registry and platform helpers
+├── config.ts       — Layered configuration resolution (flags → env → file → PATH)
+├── detect.ts       — Client discovery: binary lookup and version extraction
+├── dispatch.ts     — Prompt-to-client resolution (explicit flag → keywords → default)
+├── execute.ts      — Synchronous subprocess spawning, stdout/stderr capture, timeout handling
+└── jobs.ts         — Async job lifecycle: detached spawn, disk-backed state, polling API
 ```

 ### Responsibilities

 | Module | Responsibility |
 |---|---|
-| `cli.ts` | Parses `argv` with `minimist`, routes to `list` / `run` / `dispatch`, prints JSON or text output, and controls the process exit code. |
-| `types.ts` | Defines `ClientName`, `ClientInfo`, `ExecResult`, `ToolConfig`, and the error hierarchy (`ClientNotFoundError`, `ExecError`). |
+| `cli.ts` | Parses `argv` with `minimist`, routes to all commands, prints JSON or text output, and controls the process exit code. |
+| `cli-helpers.ts` | Shared helpers for `reportError`, `reportCliError`, `handleSyncRun`, and `handleAsyncRun` to keep `cli.ts` focused on routing. |
+| `types.ts` | Defines `ClientName`, `ClientInfo`, `ExecResult`, `ToolConfig`, `Job`, `JobRecord`, `JobStatus`, and the error hierarchy (`ClientNotFoundError`, `ExecError`, `JobNotFoundError`, `JobResultUnavailableError`). |
 | `constants.ts` | Holds the canonical `CLIENT_NAMES` array and `isWindows()` helper used by discovery and config. |
 | `config.ts` | Resolves per-client binary paths and the optional `defaultClient` from four layered sources. |
 | `detect.ts` | Locates each client binary on `PATH`, falls back to a manual directory scan, and invokes `--version` to extract a semver string. |
 | `dispatch.ts` | Chooses the target client from a prompt string using ordered keyword matching, with overrides for explicit `--client` and `defaultClient`. |
 | `execute.ts` | Spawns the chosen client with its native argument shape, buffers `stdout`/`stderr`, enforces a timeout, and returns an `ExecResult` or throws a typed error. |
+| `jobs.ts` | Manages background jobs: writes job records to disk, spawns detached child processes, tracks running children in memory, and provides `status`, `results`, `cancel`, `list`, and `cleanup` operations. |

 ## Data Flow

-A typical `dispatch` invocation flows through four stages:
+### Synchronous dispatch (`run --sync`, `dispatch --sync`)
+
+A sync invocation flows through four stages:

 ```
 ┌─────────────┐     ┌─────────────┐     ┌─────────────┐     ┌─────────────┐
@@ -41,6 +47,30 @@ A typical `dispatch` invocation flows through four stages:
  --version                               fallback default     timeout / exitCode
 ```

+### Asynchronous dispatch (`run`, `dispatch`, `start`)
+
+An async invocation adds the `jobs.ts` stage. The caller receives a job ID immediately; the child process continues in the background.
+
+```
+┌─────────────┐     ┌─────────────┐     ┌─────────────┐     ┌─────────────┐     ┌─────────────┐
+│   detect    │ ──► │   config    │ ──► │  dispatch   │ ──► │   execute   │ ──► │    jobs     │
+└─────────────┘     └─────────────┘     └─────────────┘     └─────────────┘     └─────────────┘
+     │                    │                    │                    │                    │
+  which/where        flags/env/file       keyword scan         spawn child       write job file
+  PATH walk          defaultClient        --client override    capture output    detached + unref
+  --version                               fallback default     timeout / exitCode  update on close
+```
+
+Later, lifecycle commands read from or modify the job store:
+
+```
+  status <jobId>  ──►  readJobFile  ──►  return Job (sans stdout/stderr)
+  results <jobId> ──►  readJobFile  ──►  return ExecResult (completed only)
+  cancel <jobId>  ──►  readJobFile  ──►  kill child or PID  ──►  write cancelled status
+  list-jobs       ──►  readdir jobDir ──►  read each file  ──►  sort + filter
+  cleanup-jobs    ──►  readdir jobDir ──►  stat mtime       ──►  unlink old files
+```
+
 ### 1. Detect

 `detectClients()` iterates over `CLIENT_NAMES` and attempts to locate each binary:
@@ -55,9 +85,9 @@ Result: an array of `ClientInfo` objects with `name`, `found`, `path`, and `vers

 `resolveConfig()` builds a `ResolvedConfig` by layering sources (highest to lowest precedence):

-1. **CLI flags** — `--codex-path`, `--claude-path`, `--opencode-path`, `--default-client`
+1. **CLI flags** — `--codex-path`, `--claude-path`, `--opencode-path`, `--default-client`, `--timeout`
 2. **Environment variables** — `AI_CLI_CODEX_PATH`, `AI_CLI_CLAUDE_PATH`, `AI_CLI_OPENCODE_PATH`, `AI_CLI_DEFAULT_CLIENT`
-3. **Config file** — `~/.openclaw/ai-cli-dispatch.json` (`paths` and `defaultClient` keys)
+3. **Config file** — `~/.openclaw/ai-cli-dispatch.json` (`paths`, `defaultClient`, `timeout` keys)
 4. **PATH discovery** — `which`/`where` fallback via `defaultWhichSync()`

 Only values for the three known `ClientName` entries are accepted; unknown `defaultClient` values are ignored.
@@ -78,30 +108,88 @@ This ordering intentionally prioritizes `"open code"` before `"opencode"` so the

 ### 4. Execute

-`executePrompt(client, prompt, options)` runs the selected client:
+`executePrompt(client, prompt, options)` runs the selected client synchronously:

 1. Reject empty or whitespace-only prompts with `ExecError`.
 2. Validate that an explicit `clientPath` exists on disk (if provided).
 3. Map the client to its native argument array via `CLIENT_ARGS`:
-   - `codex` → `["exec", prompt]`
-   - `claude` → `["-p", prompt]`
-   - `opencode` → `[prompt]`
+   - `codex` → `["exec", "--yolo", prompt]`
+   - `claude` → `["-p", prompt, "--dangerously-skip-permissions"]`
+   - `opencode` → `["run", "--dangerously-skip-permissions", prompt]`
 4. `spawn()` the process with `shell: false`.
 5. Buffer `stdout` and `stderr` via `"data"` listeners.
 6. Start a `setTimeout`; if it fires, `child.kill()` is sent.
-7. On `close`, resolve with `{ stdout, stderr, exitCode }`.
+7. On `close`, resolve with `{ stdout, stderr, exitCode, client, durationMs }`.
 8. On `error`, reject with `ClientNotFoundError` for `ENOENT` or `ExecError` for anything else.
 9. On timeout, reject with `ExecError` containing the buffered output so far.
+10. If `debug` is enabled, emit a `DebugInfo` object via `onDebug`.

-The default timeout is **5 minutes** (`300_000` ms).
+The default timeout is **10 minutes** (`600_000` ms).
+
+### 5. Jobs
+
+`startJob(client, prompt, options)` launches a background job:
+
+1. Generate a UUID for the job ID.
+2. Build the client argument array via `CLIENT_ARGS`.
+3. `spawn()` the process with `detached: true` and `stdio: ["ignore", "pipe", "pipe"]`.
+4. Write an initial `JobRecord` to `~/.openclaw/ai-cli-dispatch/jobs/<jobId>.json` with status `running`.
+5. Update the record with the child `pid` once available.
+6. Register the child in an in-memory `runningChildren` Map for cancellation and timeout tracking.
+7. Buffer `stdout`/`stderr` via `"data"` listeners.
+8. On `close`, finalize the record: write status (`completed`, `failed`, `timed_out`, or `cancelled`), capture stdout/stderr, and record `durationMs`.
+9. Call `child.unref()` so the dispatcher process can exit without waiting for the child.
+
+`getJob(jobId)` reads the job file and returns a `Job` (omitting the full stdout/stderr buffers).
+
+`getJobResult(jobId)` returns the `ExecResult` for a completed job.
+
+`cancelJob(jobId)` looks up the running child in memory, sends `SIGTERM`, and writes a `cancelled` status. If the child is no longer in memory, it attempts `process.kill(pid, "SIGTERM")` as a fallback.
+
+`listJobs({ filter })` reads all `.json` files in the job directory, parses them, sorts by `startedAt` descending, and optionally filters by status.
+
+`cleanupJobs({ maxAgeMs })` deletes job files whose `mtime` exceeds the threshold. Default max age is 24 hours.

 ## Design Decisions

+### Async-First Architecture
+
+The default execution mode is **async** (background job). Synchronous execution requires an explicit `--sync` flag.
+
+**Rationale:**
+- **Primary use case alignment:** Most AI CLI tasks (refactoring, test generation, migration) run for multiple minutes. Blocking the caller for that long is often undesirable in automation and orchestration contexts.
+- **Resilience:** A detached background job survives an unexpected dispatcher exit. The caller can reconnect later via `status` and `results`.
+- **Batching:** Multiple jobs can be started in parallel without blocking the dispatcher process.
+- **Backward compatibility path:** `--sync` preserves the original one-shot behavior for callers that need it, without changing the default.
+
+### Disk-Backed Job Store
+
+Job state is persisted as JSON files on disk rather than kept solely in memory.
+
+**Rationale:**
+- **Durability across restarts:** If the dispatcher process crashes or the host reboots, job files remain. A caller can still query `status` or `results` after recovery.
+- **No memory leaks:** Long-running or forgotten jobs do not accumulate in heap. Cleanup is explicit via `cleanup-jobs`.
+- **External observability:** Operators can inspect `~/.openclaw/ai-cli-dispatch/jobs/` directly without calling the CLI.
+- **Simplicity:** A file-per-job model avoids the need for an embedded database or external service. It maps cleanly to the Node.js `fs` API and is trivial to mock in tests.
+
+**Trade-off:** High-frequency job creation could strain the filesystem, but the expected volume is low (tens to hundreds of jobs, not thousands per second).
+
+### Detached-Process Approach
+
+Async jobs use `detached: true` with `child.unref()`.
+
+**Rationale:**
+- **Parent independence:** The dispatcher can start a job and exit immediately. This is essential for CLI usage where the user or orchestrator should not hold a shell open for the duration of the task.
+- **Signal isolation:** A detached process group means the child does not receive `SIGINT` or `SIGHUP` sent to the dispatcher terminal session.
+- **PID tracking:** Even though the child is detached, the `pid` is captured and written to the job file. This enables `cancelJob` to send signals even if the dispatcher has restarted and lost its in-memory `runningChildren` map.
+
+**Trade-off:** The child is truly independent. If the host reboots, the child is lost (same as any other process). The job file will eventually reflect `timed_out` or remain `running` until `cancel` or `cleanup` is run.
+
 ### Coexistence with ACP

 `ai-cli-dispatch` is intentionally **not** an ACP agent. It is a thin, local subprocess wrapper with no session state, no thread binding, and no orchestrator protocol.

- Use `ai-cli-dispatch` when you need a quick, one-shot CLI execution on the gateway host.
+- Use `ai-cli-dispatch` when you need a quick, one-shot CLI execution or a background job on the gateway host.
 - Use ACP (`docs/openclaw-acp-orchestration.md`) when you need session-bound coding harnesses, multi-turn review, or orchestrator-managed verification gates.

 This separation keeps the dispatcher small and avoids duplicating ACP’s scheduling, context persistence, and review-loop responsibilities.
@@ -118,17 +206,6 @@ Client resolution uses deterministic substring matching instead of natural-langu

 The trade-off is that prompts like `"compare codex and claude"` resolve to `codex` because `"codex"` is checked first. Users can always override with `--client`.

-### Sync-Only Initial Release
-
-The current implementation is entirely synchronous from the caller’s perspective: `executePrompt` returns a promise that resolves only when the child process exits or the timeout fires.
-
-**Rationale:**
- The primary use case is one-shot tasks (refactor, add tests, migrate) where the agent needs the complete output before proceeding.
- Streaming would require a different output contract (callbacks, generators, or an event emitter) and complicates the JSON error model.
- ACP already covers interactive, streaming, and session-based use cases.
-
-Streaming is an intentional future extension point (see below).
-
 ### Error Taxonomy

 All runtime failures are represented as typed errors so callers and tests can branch precisely:
@@ -136,7 +213,9 @@ All runtime failures are represented as typed errors so callers and tests can br
 | Error | When thrown | Data carried |
 |---|---|---|
 | `ClientNotFoundError` | Binary not on `PATH`, explicit `clientPath` missing, or `ENOENT` from `spawn` | `message` with client name |
-| `ExecError` | Empty prompt, unknown client, timeout, non-`ENOENT` spawn error, or child exit | `message` + full `ExecResult` (`stdout`, `stderr`, `exitCode`) |
+| `ExecError` | Empty prompt, unknown client, timeout, non-`ENOENT` spawn error, or child exit | `message` + full `ExecResult` (`stdout`, `stderr`, `exitCode`, `client`, `durationMs`) |
+| `JobNotFoundError` | Job ID not found in the job store | `message` with job ID |
+| `JobResultUnavailableError` | `results` called on a non-completed job | `message` with job ID and current status |

 `ExecError` carries the `ExecResult` so that timeout handlers still return partial output. This avoids losing buffered stdout/stderr when a long-running task is killed.

@@ -151,7 +230,7 @@ Every non-trivial module accepts an `options` bag with injectable dependencies (

 ### Minimal Dependency Surface

-The runtime dependency graph contains exactly one external package: `minimist` (argument parsing). Everything else uses Node.js built-ins (`child_process`, `fs`, `os`, `path`).
+The runtime dependency graph contains exactly one external package: `minimist` (argument parsing). Everything else uses Node.js built-ins (`child_process`, `fs`, `os`, `path`, `crypto`).

 **Rationale:**
 - Reduces supply-chain risk and install time.
@@ -169,7 +248,8 @@ To support a fourth (or fifth) AI CLI client, change four files in `src/` and th
 3. **`src/execute.ts`** — Add an entry to `CLIENT_ARGS` with the client’s native argument shape.
 4. **`src/config.ts`** — No change required; the existing loop over `CLIENT_NAMES` automatically picks up the new env/flag/file keys.
 5. **`src/dispatch.ts`** — Add a keyword check for the new client in `resolveClient`. Decide its precedence relative to existing keywords.
-6. **Tests** — Add colocated test cases in `tests/dispatch.test.ts`, `tests/execute.test.ts`, and `tests/detect.test.ts`.
+6. **`src/jobs.ts`** — No change required; `CLIENT_ARGS` is already shared.
+7. **Tests** — Add colocated test cases in `tests/dispatch.test.ts`, `tests/execute.test.ts`, `tests/detect.test.ts`, and `tests/jobs.test.ts`.

 No changes are needed in `cli.ts` because it iterates over `CLIENT_NAMES` for validation.

@@ -194,6 +274,8 @@ When `onData` is provided, `executePrompt` would:

 This preserves backward compatibility: existing callers that omit `onData` receive the exact same buffered `ExecResult` they get today.

+For async jobs, `jobs.ts` could store a partial `stdout`/`stderr` in the job file on each chunk (or at a throttled interval) so `status` callers can see progress without waiting for completion.
+
 ### Platform Backends

 The current Windows support is limited to discovery (`where` instead of `which`, `.exe` extension assumptions). If future clients require platform-specific spawn options (e.g., PowerShell quoting rules), the extension point is `CLIENT_ARGS` or a new `CLIENT_SPAWN_OPTIONS` record keyed by `ClientName`.
@@ -204,10 +286,12 @@ The test suite in `tests/` mirrors the `src/` structure:

 | Test file | Coverage |
 |---|---|
-| `cli.test.ts` | Argument parsing, command routing, JSON/text output modes, exit codes, error formatting |
+| `cli.test.ts` | Argument parsing, command routing, JSON/text output modes, exit codes, error formatting, sync vs async branches, all job lifecycle commands |
+| `cli-helpers.test.ts` | `reportError`, `reportCliError`, `handleSyncRun`, `handleAsyncRun` with JSON and text modes |
 | `config.test.ts` | Layered precedence of flags, env, file, and `which` fallback; malformed JSON tolerance |
 | `detect.test.ts` | `which` success/failure, PATH directory fallback, version parsing, missing binary handling |
 | `dispatch.test.ts` | Keyword matching, case insensitivity, `--client` precedence, `defaultClient` fallback, invalid flag handling |
-| `execute.test.ts` | Successful execution, stderr capture, non-zero exit codes, `ENOENT` → `ClientNotFoundError`, timeout, empty prompt rejection, special-character preservation |
+| `execute.test.ts` | Successful execution, stderr capture, non-zero exit codes, `ENOENT` → `ClientNotFoundError`, timeout, empty prompt rejection, special-character preservation, debug info emission |
+| `jobs.test.ts` | Job start, status query, result retrieval, cancellation, listing, cleanup, timeout handling, unknown client fallback, detached process behavior, in-memory vs on-disk consistency |

 All tests use injected mocks; no test spawns real client binaries or reads the real filesystem.
@@ -26,15 +26,98 @@ npm install

 ## Commands

+### `list`
+
+Discover and report all supported clients.
+
 ```bash
 scripts/ai-cli-dispatch list --json
+scripts/ai-cli-dispatch list --text
+```
+
+### `start` — start a background job
+
+Starts a detached background job and returns immediately with a job ID.
+
+```bash
+scripts/ai-cli-dispatch start --client codex --prompt "refactor this function"
+scripts/ai-cli-dispatch start --client claude --prompt "add tests for auth middleware"
+```
+
+### `run` — run a task (async by default)
+
+Runs a prompt through a named client. By default this is **async**: it starts a background job and returns the job ID immediately. Use `--sync` to block until the client finishes and return the full result.
+
+```bash
+# Async (default) — returns job ID immediately
 scripts/ai-cli-dispatch run --client codex --prompt "refactor this function"
 scripts/ai-cli-dispatch run --client claude --prompt "add tests for auth middleware"
 scripts/ai-cli-dispatch run --client opencode --prompt "migrate to ESM"
-scripts/ai-cli-dispatch dispatch "use codex to write tests"
+
+# Sync — blocks until completion and returns stdout/stderr/exitCode
+scripts/ai-cli-dispatch run --client codex --prompt "fix lint errors" --sync
 ```

-Use `--json` for machine-readable command output. Use `--sync` with `run` or `dispatch` to block until the client returns; the default is async (starts a background job and returns the job ID immediately).
+### `dispatch` — auto-resolve client and run (async by default)
+
+Auto-resolves the client from prompt keywords, then executes. By default this is **async**. Use `--sync` to block until completion.
+
+```bash
+# Async (default)
+scripts/ai-cli-dispatch dispatch "use codex to write tests"
+scripts/ai-cli-dispatch dispatch "claude: add unit tests for utils.ts"
+scripts/ai-cli-dispatch dispatch "opencode migrate to ESM"
+
+# Sync
+scripts/ai-cli-dispatch dispatch "review this PR" --client claude --sync
+```
+
+### Job lifecycle commands
+
+After starting an async job, manage it with these subcommands:
+
+```bash
+# Check job status
+scripts/ai-cli-dispatch status <job-id>
+
+# Get results (only when status is completed)
+scripts/ai-cli-dispatch results <job-id>
+
+# Cancel a running job
+scripts/ai-cli-dispatch cancel <job-id>
+
+# List all jobs, newest first
+scripts/ai-cli-dispatch list-jobs --json
+scripts/ai-cli-dispatch list-jobs --status running --json
+
+# Clean up old job files
+scripts/ai-cli-dispatch cleanup-jobs --max-age 24h
+```
+
+## Async vs Sync Mode
+
+The dispatcher is **async-first**: `run` and `dispatch` start a detached background job unless you pass `--sync`.
+
+| Mode | Behavior | When to use |
+|---|---|---|
+| **Async** (default) | Starts a detached process, returns a `jobId` immediately, and stores output on disk. | Fire-and-forget tasks, long-running jobs, parallel dispatches, or when you need to poll/check results later. |
+| **Sync** (`--sync`) | Blocks until the client subprocess exits, then returns `stdout`, `stderr`, and `exitCode` directly. | Short, interactive tasks where you need the result in the same turn. |
+
+Use `--timeout <ms>` to control how long a job may run before it is terminated (default: 10 minutes / 600,000 ms for both async and sync). Use `--debug` to emit diagnostic metadata to stderr.
+
+## Flags
+
+| Flag | Description |
+|---|---|
+| `--sync` | Run synchronously and block until the client returns. Default is async (starts a background job). |
+| `--timeout <ms>` | Timeout in milliseconds. Overrides the default and any config value. |
+| `--debug` | Emit diagnostic JSON to stderr (command, args, PID, duration, exit signal). |
+| `--json` | Output JSON (default). |
+| `--text` | Output plain text instead of JSON. |
+| `--client <name>` | Explicitly set the client (`codex`, `claude`, `opencode`). |
+| `--prompt <text>` | The prompt to send to the client. |
+| `--status <status>` | Filter `list-jobs` by status (`running`, `completed`, `failed`). |
+| `--max-age <number>[h\|m\|s\|d]` | Maximum age for `cleanup-jobs` (default unit: hours). |

 ## Client Discovery

@@ -46,12 +129,46 @@ The skill searches for the following clients in order:

 Run `list` to see which clients are installed and their resolved versions.

-## Background Jobs
+## Job Lifecycle & Storage

-For long-running or fire-and-forget tasks, use the programmatic job API or invoke `run` / `dispatch` without `--sync`:
+Async jobs run as detached child processes. Each job writes a record to disk at:
+
+```text
+~/.openclaw/ai-cli-dispatch/jobs/<jobId>.json
+```
+
+A job moves through the following statuses:
+
+| Status | Meaning |
+|---|---|
+| `running` | The client subprocess is active. |
+| `completed` | The subprocess exited with code 0. |
+| `failed` | The subprocess exited with a non-zero code. |
+| `timed_out` | The job exceeded `--timeout` and was terminated. |
+| `cancelled` | The job was cancelled via `cancel <job-id>`. |
+
+Example async workflow:
+
+```bash
+# 1. Start a job
+scripts/ai-cli-dispatch run --client codex --prompt "refactor auth module"
+# → { "jobId": "a1b2c3d4...", "client": "codex", "status": "running" }
+
+# 2. Poll status
+scripts/ai-cli-dispatch status a1b2c3d4...
+# → { "id": "a1b2c3d4...", "status": "running", ... }
+
+# 3. Get results when done
+scripts/ai-cli-dispatch results a1b2c3d4...
+# → { "stdout": "...", "stderr": "...", "exitCode": 0, "client": "codex", "durationMs": 42000 }
+```
+
+## Background Jobs (Programmatic API)
+
+For long-running or fire-and-forget tasks, use the programmatic job API:

 ```typescript
-import { startJob, getJob, cancelJob, listJobs, cleanupJobs } from "./src/jobs.js";
+import { startJob, getJob, getJobResult, cancelJob, listJobs, cleanupJobs } from "./src/jobs.js";

 // Start a detached job
 const job = await startJob("codex", "refactor auth module", { timeoutMs: 300_000 });
@@ -62,6 +179,10 @@ console.log(job.status); // "running"
 const latest = getJob(job.id);
 console.log(latest.status); // "running" | "completed" | "failed" | "timed_out" | "cancelled"

+// Get result (throws if not completed)
+const result = getJobResult(job.id);
+console.log(result.stdout, result.exitCode);
+
 // Cancel a running job
 cancelJob(job.id);

@@ -73,7 +194,7 @@ const running = listJobs({ filter: "running" });
 cleanupJobs({ maxAgeMs: 24 * 60 * 60 * 1000 });
 ```

-Job files are stored under `~/.openclaw/ai-cli-dispatch/jobs/<jobId>.json` and include stdout, stderr, exit code, and timing.
+Job files include stdout, stderr, exit code, timing, and error state.

 ## Output Rules