Step 01
Scan over real HTTP
Grades five dimensions against a versioned rubric by fetching live URLs — never trusting internal plugin state. What a crawler sees is what gets scored.
WordPress · AI & agent readiness
See your site the way an agent does.
Crawlwise grades your WordPress site over a real HTTP request, applies one-click fixes that never touch your content, and re-scans to prove the score moved.
- Reversible fixes
- Zero telemetry
- Real HTTP checks
- GPL-2.0
78 / 100
Level 4 Interactive
Dimensions
Discoverability 92
Content 78
Bot Access 67
Capabilities 58
Commerce n/a
Check log
D1 robots.txt present 4
D2 XML sitemap 4
D3 Link header 0
C1 Markdown negotiation 0
C2 /llms.txt 6
B1 Content Signals 5
B2 Explicit AI bot rules —
The shift
Your readers aren’t all human anymore.
ChatGPT, Claude, Perplexity and a growing fleet of agents visit sites to answer questions, buy things, and act for people. They don’t read your theme — they read your signals: robots rules, sitemaps, machine-readable content, capability descriptors.
Most WordPress sites emit almost none of them, and nothing in wp-admin tells you so.
GET /robots.txt 200
GET /sitemap.xml 200
GET /llms.txt 404
GET /.well-known/mcp.json 404
GET /.well-known/oauth-metadata 404
GET Accept: text/markdown 406
How it works
Scan. Fix. Verify — then keep it fixed.
A closed loop, not a one-off report. Every check fetches your site the way a crawler would, so a passing score reflects what an agent actually sees — even when a CDN or a stray robots.txt is overriding WordPress.
Scan Fix Verify Re-scan
Step 02
Apply one-click fixes
Every fix writes plugin settings and nothing else. No theme edits, no rewritten posts — so “Turn off” genuinely restores the prior state.
Step 03
Re-scan to prove it
Verify re-fetches over a live request to confirm the score actually moved. A scheduled re-scan catches drift, so what you fixed stays fixed.
What it grades
Five dimensions. Fifteen checks. One weighted score.
Each check earns points toward its dimension; dimensions roll up to an overall 0–100 score and a readiness level. Inapplicable dimensions leave the denominator, so the number always reflects your site.
ID Dimension / check Points
Discoverability Can an agent find your content and its structure at all? 11 pts
D1 robots.txt present & sane 4
D2 XML sitemap 4
D3 Link header for discovery 3
Content Is the content itself readable by a machine, not just a browser? 18 pts
C1 Markdown content negotiation 8
C2 /llms.txt 6
C3 Structured HTML 4
Bot Access Do you state, in machine terms, which agents may do what? 15 pts
B1 Content Signals in robots.txt 5
B2 Explicit AI bot rules 5
B3 Web Bot Auth key directory 5
Capabilities Can an agent discover what your site can actually do? 12 pts
K1 API catalog 3
K2 MCP card 3
K3 Agent skills 3
K4 OAuth metadata 3
Commerce Can an agent transact? WooCommerce only. 10 pts
M1 UCP signals 5
M2 x402 signals 5
Total 15 checks across 5 dimensions · points earned ÷ points possible 66
Info-tier results never score
A “couldn’t verify” result stays informational. A blocked loopback never becomes a misleading pass — or a misleading fail.
Defers to your SEO plugin
Detects Yoast, Rank Math and AIOSEO per feature, so you never end up with two sitemaps or two JSON-LD graphs.
The readiness ladder
From invisible to agent-native.
Your overall score maps to one of six levels. It’s a real progression — each rung assumes the one below it — and it doubles as a shareable badge.
Your site · 78
0–9 10–29 30–54 55–74 75–89 90–100
0 Invisible Agents can’t reliably find or read the site. 0–9
1 Discoverable Basic robots & sitemap signals are in place. 10–29
2 Readable Content is available in machine-friendly form. 30–54
3 Governed You state explicitly which bots may do what. 55–74
4 Interactive You are here Capabilities & APIs are discoverable by agents. 75–89
5 Agent-Native Fully legible and actionable — the top of the rubric. 90–100
Why Crawlwise
A scanner you can trust to fix things.
Plenty of tools flag what’s missing. Crawlwise fixes it safely, proves the fix over a live request, and stays out of your content’s way.
Reversible by design
Every fix writes plugin settings only — no theme edits, no rewritten posts. Turning a fix off restores the exact prior state.
Real HTTP verification
Checks fetch your live site, catching a CDN or a physical robots.txt that overrides WordPress — the kind of thing internal state hides.
Privacy-first
Requests hit your own site only. Nothing is sent to a third party, and there is no telemetry — not now, not opt-out-later.
Drift protection
A scheduled re-scan re-checks your score on a cadence, so a theme update or host change that regresses a signal doesn’t go unnoticed.
Plays well with SEO plugins
Detects Yoast, Rank Math and AIOSEO and defers per feature — so you never end up with two sitemaps or two JSON-LD graphs.
Agent-callable MCP server
An optional MCP server exposes scan / fix / verify / revert as tools an agent can call directly. Off by default, gated by a token you generate.
For agents, by design
Let an agent run the whole loop.
Flip on the optional MCP server and your readiness workflow becomes a set of tools an AI agent can invoke itself — scan, apply a fix, verify over HTTP, revert if needed.
Off by default Nothing is exposed until you enable it in Settings.
Bearer-token gated Stored as a SHA-256 hash, shown once, never in an export.
Hardened Origin-validated and rate-limited against failed auth per IP.
# Register Crawlwise as an MCP server $ claude mcp add --transport http crawlwise \ https://your-site.com/wp-json/crawlwise/mcp \ --header "Authorization: Bearer ••••••••••" ✓ connected · 4 tools available scan fix verify revert agent> scan, then fix everything failing # → 61 → applied D1, C2, B1 → verified → 84
Free · GPL-2.0 · WordPress 6.7+
Make your site agent-native.
Install Crawlwise, run one scan, and see exactly where you stand with the machines that are already reading the web.
One scan · reversible fixes · no account, no telemetry