Everything you need to run Crawlwise.
Install it, understand the rubric, apply reversible fixes, and — if you want — hand the whole loop to an agent over MCP.
Installation
Crawlwise requires WordPress 6.7+ and PHP 7.4+.
- 1 Upload Upload the crawlwise folder to /wp-content/plugins/, or install it through Plugins → Add New.
- 2 Activate Activate the plugin through the Plugins screen.
- 3 Scan Open Crawlwise in the admin menu and click Scan to grade your site.
- 4 Fix & verify Apply a fix from any failing check, then click Verify to confirm the change over a real HTTP request.
Activation creates one custom table ({prefix}_crawlwise_scans) for
score history and flushes rewrite rules so /llms.txt, *.md and
.well-known/ routes can resolve.
The three-beat loop
The core workflow is a three-beat loop you can repeat any time your site changes, producing an overall 0–100 score and a readiness level.
Crawlwise fetches your live URLs and grades five dimensions against the rubric.
Each failing check offers a one-click fix. Applying one writes plugin settings only — it never edits your theme or rewrites posts.
Re-fetch over a real request to confirm the score moved. A scheduled re-scan then watches for drift.
Because every check is an HTTP request against your own site, the score reflects what a
crawler actually sees — including cases where a CDN or a physical robots.txt is
overriding WordPress.
The rubric
Scoring is driven entirely by a versioned rubric (rubric-v1) — the single
source of truth for what gets scored and how much it’s worth. Dimensions roll up to the
overall score using a weighted average; info-tier results and inapplicable dimensions are
excluded from the denominator.
A new rubric version is a new file — the engine stays data-driven, so scoring rules can evolve without changing engine code.
Dimensions & checks
Fifteen checks across five dimensions. Each is graded independently over HTTP.
Fixes & reversibility
This is the part that sets Crawlwise apart from a stateless scanner: it doesn’t just tell you what’s wrong, it fixes it — safely.
Enabled static export and nothing happened? It won’t overwrite an existing
robots.txt / llms.txt. Remove or rename the existing file if you
want the plugin to manage it.
Agent (MCP) server
Crawlwise ships an optional MCP server that exposes scan, fix,
verify and revert as tools an AI agent can call directly. It is off
by default and requires a bearer token you generate yourself.
- 1 Enable Go to Crawlwise → Settings, tick Enable the MCP server, and click Save Changes.
- 2 Copy the token It is stored hashed and cannot be displayed again — regenerate it if you lose it.
- 3 Register with your agent Add the endpoint and bearer token to Claude Code, Cursor, or any MCP-capable client:
# Register Crawlwise with your agent
$ claude mcp add --transport http crawlwise \
https://your-site.com/wp-json/crawlwise/mcp \
--header "Authorization: Bearer <token>"
✓ connected · tools: scan, fix, verify, revertThe token is stored as a SHA-256 hash, never in plain text, and is excluded from settings exports.
Privacy & security
Crawlwise makes HTTP requests to your own site only. It has no telemetry — nothing is sent to us or any third party.
/llms.txt, the Markdown output and the JSON-LD graph only expose
already-public content: password-protected, private and draft content is never
included.
FAQ
Does it conflict with Yoast, Rank Math, or AIOSEO?
No. Crawlwise detects them and defers per feature — you’ll never get duplicate meta descriptions or two JSON-LD graphs.
Why does a check say “couldn’t verify”?
Some hosts block a site from making HTTP requests to itself. Crawlwise reports an informational result rather than guessing a misleading pass or fail.
I enabled static export and nothing happened.
Static export refuses to overwrite an existing robots.txt or llms.txt — your file is never clobbered. Remove or rename it if you want the plugin to manage that path.
What happens when I uninstall?
Uninstalling removes the custom table, all options and every emitted route. Your site is left exactly as Crawlwise found it.
Changelog
Every release, newest first.
/llms.txt and JSON-LD descriptions. The
MCP bearer token is stored hashed, shown once, and excluded from settings export.
*.md and .well-known/* URLs return a real 404 when a feature
is off, and scan history is capped per scope.