vs
Comparison
robots.txt vs Web Hypertext Application Technology Working Group
Who’s behind each, what their homepages track, what they cost and how often their policies changed, measured the same way for both. By what they offer, they’re close.
An informational site about web robots (crawlers and spiders) and the robots.txt exclusion standard, with an FAQ, a database of known robots, a robots.txt checker and an IP lookup tool.
Standards body that publishes and maintains the HTML Living Standard and other web specifications such as DOM, Fetch and URL, developed openly on GitHub with community contributions.
- Company
- Not stated
- Not stated
- Registered in
- Unknown
- Unknown
- Online since
- Unknown
- Unknown
- Audience
- Smaller site
- Smaller site
- Trackers on the homepage
-
-
- Ad trackers
- None
- None
- Cookie banner
- –
- –
- Prices
- No pricing page found
- No pricing page found
- Last price change
- –
- –
- Privacy policy changes
- 0
- 0
- Terms changes
- 0
- 0
- Open positions
- –
- –
Trackers are what each homepage loads before cookie consent; changes count since we started checking each site weekly. Hover a dotted value for its source and date.