133 lines
4.1 KiB
Markdown
133 lines
4.1 KiB
Markdown
# Web Automation Consolidation Implementation Plan
|
|
|
|
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
|
|
|
**Goal:** Consolidate the separate `playwright-safe` skill into `web-automation` and publish a single web skill with both one-shot extraction and broader automation.
|
|
|
|
**Architecture:** Move the proven safe one-shot extractor into `skills/web-automation/scripts/extract.js`, update `web-automation` docs to expose it as the simple path, and remove the separate `playwright-safe` skill and docs. Keep the extractor behavior unchanged except for its new location/name.
|
|
|
|
**Tech Stack:** Node.js, Playwright, Camoufox skill docs, git
|
|
|
|
---
|
|
|
|
### Task 1: Create isolated worktree
|
|
|
|
**Files:**
|
|
- Modify: repo git metadata only
|
|
|
|
**Step 1: Create worktree**
|
|
|
|
Run:
|
|
```bash
|
|
git -C /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills worktree add /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills/.worktrees/web-automation-consolidation -b feature/web-automation-consolidation
|
|
```
|
|
|
|
**Step 2: Verify baseline**
|
|
|
|
Run:
|
|
```bash
|
|
git -C /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills/.worktrees/web-automation-consolidation status --short --branch
|
|
```
|
|
Expected: clean feature branch
|
|
|
|
### Task 2: Move the extractor into web-automation
|
|
|
|
**Files:**
|
|
- Create: `skills/web-automation/scripts/extract.js`
|
|
- Read: `skills/playwright-safe/scripts/playwright-safe.js`
|
|
|
|
**Step 1: Copy the extractor**
|
|
- Copy the proven script content into `skills/web-automation/scripts/extract.js`
|
|
- Adjust only relative paths/messages if needed
|
|
|
|
**Step 2: Preserve behavior**
|
|
- Keep JSON-only output
|
|
- Keep URL validation
|
|
- Keep stealth/anti-bot behavior
|
|
- Keep sandbox enabled
|
|
|
|
### Task 3: Update skill and docs
|
|
|
|
**Files:**
|
|
- Modify: `skills/web-automation/SKILL.md`
|
|
- Modify: `docs/web-automation.md`
|
|
- Modify: `README.md`
|
|
- Modify: `docs/README.md`
|
|
- Delete: `skills/playwright-safe/SKILL.md`
|
|
- Delete: `skills/playwright-safe/package.json`
|
|
- Delete: `skills/playwright-safe/package-lock.json`
|
|
- Delete: `skills/playwright-safe/.gitignore`
|
|
- Delete: `skills/playwright-safe/scripts/playwright-safe.js`
|
|
- Delete: `docs/playwright-safe.md`
|
|
|
|
**Step 1: Update docs**
|
|
- Make `web-automation` the only published web skill
|
|
- Document `extract.js` as the one-shot extraction path
|
|
- Remove published references to `playwright-safe`
|
|
|
|
**Step 2: Remove redundant skill**
|
|
- Delete the separate `playwright-safe` skill files and doc
|
|
|
|
### Task 4: Verify behavior
|
|
|
|
**Files:**
|
|
- Test: `skills/web-automation/scripts/extract.js`
|
|
|
|
**Step 1: Missing URL check**
|
|
|
|
Run:
|
|
```bash
|
|
cd /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills/.worktrees/web-automation-consolidation && node skills/web-automation/scripts/extract.js
|
|
```
|
|
Expected: JSON error about missing URL
|
|
|
|
**Step 2: Invalid scheme check**
|
|
|
|
Run:
|
|
```bash
|
|
cd /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills/.worktrees/web-automation-consolidation && node skills/web-automation/scripts/extract.js ftp://example.com
|
|
```
|
|
Expected: JSON error about only http/https URLs allowed
|
|
|
|
**Step 3: Smoke test**
|
|
|
|
Run:
|
|
```bash
|
|
cd /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills/.worktrees/web-automation-consolidation && node skills/web-automation/scripts/extract.js https://example.com
|
|
```
|
|
Expected: JSON with title `Example Domain`, status `200`, and no sandbox-disabling flags in code
|
|
|
|
**Step 4: Reference scan**
|
|
|
|
Run:
|
|
```bash
|
|
cd /Users/stefano/.openclaw/workspace/projects/stef-openclaw-skills/.worktrees/web-automation-consolidation && rg -n "playwright-safe" README.md docs skills
|
|
```
|
|
Expected: no remaining published references, or only intentional historical plan docs
|
|
|
|
### Task 5: Commit, push, and clean up
|
|
|
|
**Files:**
|
|
- Modify: git history only
|
|
|
|
**Step 1: Commit**
|
|
|
|
Run:
|
|
```bash
|
|
git add skills/web-automation docs README.md
|
|
git commit -m "refactor: consolidate web scraping into web-automation"
|
|
```
|
|
|
|
**Step 2: Push**
|
|
|
|
Run:
|
|
```bash
|
|
git push -u origin feature/web-automation-consolidation
|
|
```
|
|
|
|
**Step 3: Merge and cleanup**
|
|
- Fast-forward or merge to `main`
|
|
- Push `main`
|
|
- Remove the worktree
|
|
- Delete the feature branch
|