# AI Agent Site Auditor - Full Notes

## Purpose
Use this service to audit public websites for agent-readiness signals. The auditor fetches public HTML, follows same-site content URLs, respects robots.txt when requested and returns JSON findings.

## API
- Endpoint: https://ai-agent-auditor.seo-experiments.net/audit
- Method: POST
- Body fields: url, max_pages, concurrency, use_sitemap, respect_robots

## Safety
- Private, loopback, link-local, multicast and reserved IP targets are blocked.
- Keep crawls modest.
- Respect robots.txt policies.

## Sources
- https://web.dev/articles/ai-agent-site-ux
- https://llmstxt.org/