AI AGENT READINESS CHECK

AI Agent Readiness Check

Check what AI agents can find in a public website’s initial HTML, robots.txt, sitemap and llms.txt.

Back to home

Astrotor reads the public page from its server.Free · No account needed

01 / ENTER THE WEBSITE ADDRESS

What the test checks

Page structure

Title, language, canonical URL, main heading and content in the initial HTML.

Crawler rules

The rules that robots.txt declares for AI crawlers on your home page.

Discovery files

The sitemap and optional llms.txt guide help readers find important pages.

Structured data

JSON-LD blocks in the initial response describe the page in a machine-readable format.

GOOD TO KNOW

A few clear answers.

Let’s make the technical terms a little simpler.

What does the AI agent readiness test check on a website?

The test examines the home page’s initial HTML, metadata, headings, main content region and JSON-LD blocks, plus robots.txt, the sitemap and llms.txt. It helps identify missing information or declared rules that can affect automated readers. It does not run the site’s JavaScript, crawl every page or ask an AI model to interpret the business. Use the findings to investigate specific issues; this is not a complete SEO audit or a live test of every agent.

How is the AI readiness score calculated, and what does it mean?

Astrotor’s 100-point checklist assigns 30 points to metadata, 25 to readable text, 20 to structured data, 10 to the sitemap and 15 to search crawler rules. The report explains the individual thresholds. A high score means more of these checks passed, not that an AI service will recommend or cite the site. llms.txt is optional and does not lower the score. If a check cannot be completed, a range is shown rather than treating an unknown result as a failure.

Does an allowed robots.txt rule mean AI bots can access my site?

Allowed means the rules read by this tool do not block that bot from the home page. It is not a successful visit by the actual bot: a firewall, login requirement or rate limit may still prevent access. Search and training bots can have different rules. Astrotor scores OAI-SearchBot, Claude-SearchBot and PerplexityBot for search access; blocking training bots does not lower this score. Change permissions to match how you want the site to be used.

What is llms.txt, and how is it different from robots.txt and a sitemap?

llms.txt is a proposed Markdown guide that introduces a site and links to useful content for AI readers. robots.txt declares crawler rules, while a sitemap lists URLs or other sitemaps for discovery. These files serve different purposes and do not replace one another. Astrotor checks the root llms.txt for a title, summary, sections and links. Publishing it is optional here; its presence does not prove that every AI service reads it or that the linked content is accurate.

Why can the AI readiness test find no readable content?

Zero means this reader extracted no text from the initial HTML. Content loaded later by JavaScript or excluded by the extraction method may still be visible to visitors, so zero does not prove the site is empty or unreadable by every AI agent. Check the extracted text alongside the page source. The word and character counts describe only this extract; no AI model is used. For long pages, the report states when the text limit is reached.

KEEP EXPLORING