Skip to main content
ScanGov
Standards
Project
Docs
ScanGov Standards
Guidance
Government digital experience standards
Based on public policy, web protocol, guidelines and best practices.
Copy link
Share
LinkedIn
Bluesky
X
Facebook
All
Accessibility
AI-friendly
Content
Domain
Performance
Security
SEO
Social
Indicators
Standard
Why
Guidance
Crawlable
Site is available for indexing by well-behaved agents.
Allows AI bots to find and index your pages, helping your content appear in search results and reach more people.
21st Century Integrated Digital Experience Act
Memorandum (M-23-22)
The Web Robots Pages
Content available in document
Page main content is available in initial document.
Ensures search engines can read and index the content within a document, making it discoverable and improving visibility online.
The Web Robots Pages
Feedback