Purpose says what a crawler’s request is for. A search crawler builds the index an answer searches. A crawler asked by a person fetches a page while an assistant answers somebody. A training crawler collects pages for future models.
Addresses published means the operator publishes the addresses its crawler sends from, so a logged request can be verified. For the others, a request can only claim the name.
Agent Analytics
AI crawlers
Every AI crawler that the crawler audit and the crawler logs know, and what each one feeds.
The Crawlers tab reads robots.txt for each crawler below, and it counts the requests of those that send any from the site’s logs. A crawler that feeds no tracked platform feeds an assistant or a model that FullReach AI does not measure.