Purpose says what a crawler’s request is for. A search crawler builds the index an answer searches. A crawler asked by a person fetches a page while an assistant answers somebody. A training crawler collects pages for future models.
Addresses published means the operator publishes the addresses its crawler sends from, so a logged request can be verified. For the others, a request can only claim the name.
Website
AI crawlers
Every AI crawler that the crawler audit and the crawler logs know, and what each one feeds.
The Crawlability audit reads robots.txt for each crawler below, and Crawler Visits counts the requests of those that send any. A crawler that feeds no tracked platform feeds an assistant or a model that FullReach AI does not measure.