Crawlers and Bots

The following page details network ranges, IPs, and points-of-presence (PoPs) for well known crawlers and bots. We have also provided a list of the top crawler/bot hosting providers.

The crawler/bot data shown here is available in our Netify Data Feeds solution. The data is accessible via our IP API as well as downloadable datasets.

Get Started

We provide network intelligence to help deliver better digital experiences to end users and businesses. Use our APIs and datasets to get started today.

Subscription Plans

Crawlers and Bots

The Netify data feed provides information on the following crawlers/bots:

Crawler/Bot
AhrefsBot - Logo AhrefsBot
Applebot - Logo Applebot
Babbar Bot - Logo Babbar Bot
Bingbot - Logo Bingbot
Censys Scanner - Logo Censys Scanner
ChatGPT Bot - Logo ChatGPT Bot
DataForSEO Bot - Logo DataForSEO Bot
Googlebot - Logo Googlebot
MajesticBot - Logo MajesticBot
Onyphe Bot - Logo Onyphe Bot
OpenAI SearchBot - Logo OpenAI SearchBot
PetalBot - Logo PetalBot
Seekport Bot - Logo Seekport Bot
Semrush Bot - Logo Semrush Bot
Shadowserver Bot - Logo Shadowserver Bot
Yandex Bot - Logo Yandex Bot

Crawler/Bot Hosting - Top 50

The following is a list of the top crawler/bot hosting providers based on the number of different crawlers/bots. Hetzner is the most prevalent provider.

Host Apps IPs
Hetzner - Logo Hetzner 3 425
DigitalOcean - Logo DigitalOcean 2 71
OVHcloud - Logo OVHcloud 2 565
Google Cloud Platform - Logo Google Cloud Platform 2 363
Scaleway - Logo Scaleway 2 1,040
Microsoft Azure - Logo Microsoft Azure 1 4,072
Vultr - Logo Vultr 1 61
Zenlayer - Logo Zenlayer 1 2
Huawei Cloud - Logo Huawei Cloud 1 1,412
Hurricane Electric - Logo Hurricane Electric 1 883
Leaseweb - Logo Leaseweb 1 8

Integration and Custom Solutions

Do you have any questions about integration, APIs or custom development?

Contact Us