AI Tools
Can AI agents actually see your website? Test llms.txt, AI crawler access (GPTBot, ClaudeBot, PerplexityBot…), MCP, structured data and JavaScript-free content — 13 checks, graded A–F in seconds.
Whether AI systems can read your site at all: robots.txt rules for 9 major AI crawlers, content that survives without JavaScript, and no challenge walls turning agents away.
Whether AI systems can understand and cite you accurately: JSON-LD structured data, an XML sitemap, and a machine-readable feed of what’s new.
The emerging agentic-web standards: llms.txt, agents.txt, MCP server manifests, OpenAPI specs, Markdown content negotiation and x402 machine payments.
It measures how well a website works for AI systems — crawlers that power AI answers, assistants that cite sources, and autonomous agents that browse on behalf of users. We check 13 signals across crawlability, discovery and AI specs, then grade the result A–F out of 100.
llms.txt is a plain-text file at your site’s root (like robots.txt) that gives language models a curated map: what your site is, which pages matter, and how you want to be described. It takes minutes to add and stops AI systems from guessing.
Depends on your business. Blocking keeps content out of AI training and AI answers — which also makes you invisible where a growing share of discovery happens. Many sites allow citation crawlers and block pure training bots. Whatever you choose, it should be deliberate — and monitored, because a deploy can silently rewrite robots.txt.
An open protocol that lets AI assistants connect to services as tools — reading data and taking actions with the user’s permission. A site with an MCP manifest isn’t just visible to AI; it’s usable by it.
Most AI crawlers and agents don’t execute JavaScript — they read the raw HTML your server returns. If your page renders everything client-side, AI systems see a blank shell. Server-side rendering or pre-rendering fixes it (and helps SEO and performance too).
Highest impact first: unblock the AI crawlers you actually want, serve real content without JavaScript, add JSON-LD structured data, publish a sitemap and RSS feed, and add llms.txt. If you offer an API, publish an OpenAPI spec and MCP manifest so agents can use it directly. Then monitor it — these signals break silently during deploys.