Crawlers
Crawlers and Bots
The following page details network ranges, IPs, and points-of-presence (PoPs) for well known crawlers and bots. We have also provided a list of the top crawler/bot hosting providers.
The crawler/bot data shown here is available in our Netify Data Feeds solution. The data is accessible via our IP API as well as downloadable datasets.
Get Started
We provide network intelligence to help deliver better digital experiences to end users and businesses. Use our APIs and datasets to get started today.
Subscription PlansCrawlers and Bots
The Netify data feed provides information on the following crawlers/bots:
Crawler/Bot Hosting - Top 50
The following is a list of the top crawler/bot hosting providers based on the number of different crawlers/bots. Scaleway is the most prevalent provider.
| Host | Apps | IPs |
|---|---|---|
Scaleway |
3 | 1,041 |
Hetzner |
3 | 639 |
DigitalOcean |
2 | 192 |
OVHcloud |
2 | 779 |
Google Cloud Platform |
2 | 364 |
Amazon AWS |
1 | 425 |
Microsoft Azure |
1 | 4,138 |
Vultr |
1 | 175 |
Zenlayer |
1 | 2 |
Huawei Cloud |
1 | 1,469 |
Meta CDN |
1 | 1,858 |
Hurricane Electric |
1 | 884 |
Leaseweb |
1 | 8 |
THG Ingenuity |
1 | 189 |
Netify Resources
Integration and Custom Solutions
Do you have any questions about integration, APIs or custom development?
Contact Us