Resources
Bot directory: search engines, AI crawlers and agents we recognise
This is the list of 142 named bots that FingerPrint Checker recognises: search engines, AI crawlers, AI assistants and agents, SEO tools, uptime monitors and link-preview fetchers. Each entry says who runs the bot, what it is for and whether we can confirm that a request really comes from that operator. You can allow or block any of them by name, or by category, in your project rules.
How to read a label
Every entry carries one of three labels. They tell you how much a customer can rely on the name.
- Verified: we can confirm the operator.
- Signed: requests carry a cryptographic signature from the operator.
- Unknown: identified by its self-description only.
What happens to a request
A request that matches an entry and passes its label is reported as a good bot. A request that claims a name we can confirm but fails the confirmation is reported as spoofed, and counts as a bad bot. A bot whose label is Unknown is reported as a bad bot, because anyone can type its name.
Events and webhooks carry the bot name, its provider, its category and the label, so you can see what visited and decide what to do about it.
Showing 142 of 142 bots.
Search crawler
| Bot | Operator | Label |
|---|---|---|
| Googlebot | Verified | |
| Storebot-Google | Verified | |
| Google-InspectionTool | Verified | |
| GoogleOther | Verified | |
| Bingbot | Microsoft | Verified |
| BingPreview | Microsoft | Verified |
| Yahoo Slurp | Yahoo | Unknown |
| DuckDuckBot | DuckDuckGo | Verified |
| Baiduspider | Baidu | Verified |
| YandexBot | Yandex | Verified |
| YandexImages | Yandex | Verified |
| Applebot | Apple | Verified |
| Sogou Spider | Sogou | Unknown |
| Naver Yeti | Naver | Unknown |
| Daumoa | Kakao | Unknown |
| SeznamBot | Seznam.cz | Unknown |
| MojeekBot | Mojeek | Unknown |
| Qwantify | Qwant | Unknown |
| CocCocBot | Coc Coc | Unknown |
| PetalBot | Huawei | Unknown |
| Mail.ru Bot | VK | Unknown |
| 360Spider | Qihoo 360 | Unknown |
| Bravebot | Brave | Unknown |
AI crawler
| Bot | Operator | Label |
|---|---|---|
| Google-CloudVertexBot | Verified | |
| GPTBot | OpenAI | Verified |
| ClaudeBot | Anthropic | Verified |
| anthropic-ai | Anthropic | Verified |
| CCBot | Common Crawl | Verified |
| Bytespider | ByteDance | Unknown |
| TikTokSpider | ByteDance | Unknown |
| Amazonbot | Amazon | Unknown |
| Meta-ExternalAgent | Meta | Unknown |
| FacebookBot | Meta | Unknown |
| cohere-training-data-crawler | Cohere | Unknown |
| Diffbot | Diffbot | Unknown |
| AI2Bot | Allen Institute for AI | Unknown |
| Timpibot | Timpi | Unknown |
| ImagesiftBot | ImageSift | Unknown |
| ICC-Crawler | NICT | Unknown |
| SBIntuitionsBot | SB Intuitions | Unknown |
| Cotoyogi | ROIS | Unknown |
| Factset_spyderbot | FactSet | Unknown |
| VelenPublicWebCrawler | Velen Crawler | Unknown |
AI search
| Bot | Operator | Label |
|---|---|---|
| OAI-SearchBot | OpenAI | Verified |
| Claude-SearchBot | Anthropic | Verified |
| PerplexityBot | Perplexity | Verified |
| Amzn-SearchBot | Amazon | Unknown |
| Meta-WebIndexer | Meta | Unknown |
| YouBot | You.com | Unknown |
| PhindBot | Phind | Unknown |
| Andibot | Andi | Unknown |
| Kimi-SearchBot | Moonshot AI | Unknown |
| ExaSearchBot | Exa | Unknown |
AI assistant
| Bot | Operator | Label |
|---|---|---|
| Google-NotebookLM | Verified | |
| ChatGPT-User | OpenAI | Verified |
| Claude-User | Anthropic | Verified |
| Perplexity-User | Perplexity | Verified |
| DuckAssistBot | DuckDuckGo | Unknown |
| Amzn-User | Amazon | Unknown |
| Meta-ExternalFetcher | Meta | Unknown |
| cohere-ai | Cohere | Unknown |
| MistralAI-User | Mistral AI | Unknown |
AI agent
| Bot | Operator | Label |
|---|---|---|
| ChatGPT agent | OpenAI | Signed |
| AmazonBuyForMe | Amazon | Unknown |
SEO
| Bot | Operator | Label |
|---|---|---|
| SemrushBot | Semrush | Unknown |
| SemrushBot-SWA | Semrush | Unknown |
| SemrushBot-OCOB | Semrush | Unknown |
| SiteAuditBot | Semrush | Unknown |
| AhrefsBot | Ahrefs | Unknown |
| AhrefsSiteAudit | Ahrefs | Unknown |
| MJ12bot | Majestic | Unknown |
| DotBot | Moz | Unknown |
| Rogerbot | Moz | Unknown |
| Screaming Frog SEO Spider | Screaming Frog | Unknown |
| DataForSeoBot | DataForSEO | Unknown |
| BLEXBot | WebMeUp | Unknown |
| SEOkicks | SEOkicks | Unknown |
| SerpstatBot | Serpstat | Unknown |
| SISTRIX Crawler | SISTRIX | Unknown |
| MegaIndex | MegaIndex | Unknown |
| Barkrowler | Babbar | Unknown |
| ContentKing | Conductor | Unknown |
Monitoring
| Bot | Operator | Label |
|---|---|---|
| Chrome Lighthouse | Unknown | |
| UptimeRobot | UptimeRobot | Unknown |
| Pingdom | SolarWinds | Unknown |
| StatusCake | StatusCake | Unknown |
| Site24x7 | Zoho | Unknown |
| HetrixTools | HetrixTools | Unknown |
| Uptime Kuma | Uptime Kuma | Unknown |
| GTmetrix | GTmetrix | Unknown |
| Datadog Synthetics | Datadog | Unknown |
| Checkly | Checkly | Unknown |
| Siteimprove | Siteimprove | Unknown |
| Better Stack | Better Stack | Unknown |
Link preview
| Bot | Operator | Label |
|---|---|---|
| facebookexternalhit | Meta | Unknown |
| Twitterbot | X | Unknown |
| LinkedInBot | Unknown | |
| Slackbot | Slack | Unknown |
| Discordbot | Discord | Unknown |
| TelegramBot | Telegram | Unknown |
| Meta | Unknown | |
| Pinterestbot | Unknown | |
| Redditbot | Unknown | |
| SkypeUriPreview | Microsoft | Unknown |
| Embedly | Embedly | Unknown |
| Iframely | Iframely | Unknown |
| Mastodon | Mastodon | Unknown |
| VKShare | VK | Unknown |
| Quora Link Preview | Quora | Unknown |
| bitlybot | Bitly | Unknown |
Advertising
| Bot | Operator | Label |
|---|---|---|
| AdIdxBot | Microsoft | Unknown |
| AdsBot-Google | Verified | |
| Mediapartners-Google | Verified | |
| OAI-AdsBot | OpenAI | Unknown |
| AmazonAdBot | Amazon | Unknown |
| YandexDirect | Yandex | Verified |
| CriteoBot | Criteo | Unknown |
Aggregator
| Bot | Operator | Label |
|---|---|---|
| FeedFetcher-Google | Verified | |
| Feedly | Feedly | Unknown |
| Unknown | ||
| Inoreader | Inoreader | Unknown |
| NewsBlur | NewsBlur | Unknown |
| Feedbin | Feedbin | Unknown |
| The Old Reader | The Old Reader | Unknown |
| Feedspot | Feedspot | Unknown |
| BazQux Reader | BazQux | Unknown |
| Miniflux | Miniflux | Unknown |
| FreshRSS | FreshRSS | Unknown |
| Omgili | Webz.io | Unknown |
Scraping
| Bot | Operator | Label |
|---|---|---|
| Scrapy | Zyte | Unknown |
| HTTrack | HTTrack | Unknown |
| img2dataset | img2dataset | Unknown |
| Crawl4AI | Crawl4AI | Unknown |
Other
| Bot | Operator | Label |
|---|---|---|
| APIs-Google | Verified | |
| Google-Read-Aloud | Verified | |
| Google-Site-Verifier | Verified | |
| archive.org_bot | Internet Archive | Unknown |
| Heritrix | Internet Archive | Unknown |
| W3C Validator | W3C | Unknown |
| NetcraftSurveyAgent | Netcraft | Unknown |
| CensysInspect | Censys | Unknown |
| Google Favicon | Unknown |
Some entries draw on ai.robots.txt (MIT licence). ai.robots.txt on GitHub
Frequently asked questions
Can I allow Googlebot but block AI crawlers?
Yes. Add one rule that allows the bot by name and another that denies the AI crawler category. The first rule that matches decides, so put the allow rule first. The "block crawlers" request filter runs before rules and blocks self-declared crawlers, including Googlebot, so leave it off if you want rules to allow Googlebot.
Why is a bot with the Unknown label treated as bad?
Its name can be copied by anyone, so we cannot tell the real bot from an imitation. If you trust a particular one, write a rule that allows it by name.
How often is the list updated?
The list shown here was last synced on 2026-10-27. New bots are added as they appear.
Is a bot I do not see in the list blocked?
No. A bot that is not in the directory is judged on how the visitor behaves, like any other visitor.
Try it on your own sign-up flow
Every new account gets 2,000 free checks. No card required.