feat: add safe Playwright scraper skill
This commit is contained in:
@@ -6,6 +6,7 @@ This folder contains detailed docs for each skill in this repository.
|
||||
|
||||
- [`elevenlabs-stt`](elevenlabs-stt.md) — Local audio transcription through ElevenLabs Speech-to-Text
|
||||
- [`gitea-api`](gitea-api.md) — REST-based Gitea automation (no `tea` CLI required)
|
||||
- [`playwright-safe`](playwright-safe.md) — Single-entry Playwright scraper for one-shot extraction with JS rendering and moderate anti-bot handling
|
||||
- [`portainer`](portainer.md) — Portainer stack management (list, lifecycle, updates, image pruning)
|
||||
- [`searxng`](searxng.md) — Privacy-respecting metasearch via a local or self-hosted SearXNG instance
|
||||
- [`web-automation`](web-automation.md) — Playwright + Camoufox browser automation and scraping
|
||||
|
||||
72
docs/playwright-safe.md
Normal file
72
docs/playwright-safe.md
Normal file
@@ -0,0 +1,72 @@
|
||||
# playwright-safe
|
||||
|
||||
Single-entry Playwright scraper for one-shot page extraction with JavaScript rendering and moderate anti-bot handling.
|
||||
|
||||
## What this skill is for
|
||||
|
||||
- Extracting title, visible text, and metadata from one URL
|
||||
- Pages that need client-side rendering
|
||||
- Moderate anti-bot shaping without a full browser automation workflow
|
||||
- Structured JSON output that agents can consume directly
|
||||
|
||||
## What this skill is not for
|
||||
|
||||
- Multi-step browser workflows
|
||||
- Authenticated login flows
|
||||
- Interactive click/type sequences across multiple pages
|
||||
|
||||
Use `web-automation` for those broader browser tasks.
|
||||
|
||||
## Runtime requirements
|
||||
|
||||
- Node.js 18+
|
||||
- Local Playwright install under the skill directory
|
||||
|
||||
## First-time setup
|
||||
|
||||
```bash
|
||||
cd ~/.openclaw/workspace/skills/playwright-safe
|
||||
npm install
|
||||
npx playwright install chromium
|
||||
```
|
||||
|
||||
## Entry point
|
||||
|
||||
```bash
|
||||
node skills/playwright-safe/scripts/playwright-safe.js "<URL>"
|
||||
```
|
||||
|
||||
Only pass a user-provided `http` or `https` URL.
|
||||
|
||||
## Options
|
||||
|
||||
```bash
|
||||
WAIT_TIME=5000 node skills/playwright-safe/scripts/playwright-safe.js "<URL>"
|
||||
SCREENSHOT_PATH=/tmp/page.png node skills/playwright-safe/scripts/playwright-safe.js "<URL>"
|
||||
SAVE_HTML=true node skills/playwright-safe/scripts/playwright-safe.js "<URL>"
|
||||
HEADLESS=false node skills/playwright-safe/scripts/playwright-safe.js "<URL>"
|
||||
USER_AGENT="Mozilla/5.0 ..." node skills/playwright-safe/scripts/playwright-safe.js "<URL>"
|
||||
```
|
||||
|
||||
## Output
|
||||
|
||||
The script prints JSON only. It includes:
|
||||
|
||||
- `requestedUrl`
|
||||
- `finalUrl`
|
||||
- `title`
|
||||
- `content`
|
||||
- `metaDescription`
|
||||
- `status`
|
||||
- `elapsedSeconds`
|
||||
- `challengeDetected`
|
||||
- optional `screenshot`
|
||||
- optional `htmlFile`
|
||||
|
||||
## Security posture
|
||||
|
||||
- Keeps lightweight stealth and anti-bot shaping
|
||||
- Keeps the browser sandbox enabled
|
||||
- Does not use `--no-sandbox`
|
||||
- Does not use `--disable-setuid-sandbox`
|
||||
- Avoids site-specific extractors and cross-skill dependencies
|
||||
@@ -9,6 +9,11 @@ Automated web browsing and scraping using Playwright with Camoufox anti-detectio
|
||||
- Extracting page content to markdown
|
||||
- Working with bot-protected or dynamic pages
|
||||
|
||||
## Routing rule
|
||||
|
||||
- For one-shot page extraction from a single URL, prefer `playwright-safe`
|
||||
- Use `web-automation` only when the task needs interactive browser control, multi-step navigation, or authenticated flows
|
||||
|
||||
## Requirements
|
||||
|
||||
- Node.js 20+
|
||||
|
||||
Reference in New Issue
Block a user