db

Robots Database

The Robots Database lists robot software implementations and operators.
Robots listed here have been submitted by their owners, or by web site owners who have been visited by the robots. A listing here does not mean that a robot is endorsed in any way.
For a list of User-Agents (including bots) in the wild, see www.botsvsbrowsers.com
This robots database is currently undergoing re-engineering. Due to poular demand we have restored the existing data, but addition/modification are disabled.
  1. ABCdatos BotLink
  2. Acme.Spider
  3. Ahoy! The Homepage Finder
  4. Alkaline
  5. Anthill
  6. Walhello appie
  7. Arachnophilia
  8. Arale
  9. Araneo
  10. AraybOt
  11. ArchitextSpider
  12. Aretha
  13. ARIADNE
  14. arks
  15. AskJeeves
  16. ASpider (Associative Spider)
  17. ATN Worldwide
  18. Atomz.com Search Robot
  19. AURESYS
  20. BackRub
  21. Bay Spider
  22. Big Brother
  23. Bjaaland
  24. BlackWidow
  25. Die Blinde Kuh
  26. Bloodhound
  27. Borg-Bot
  28. BoxSeaBot
  29. bright.net caching robot
  30. BSpider
  31. CACTVS Chemistry Spider
  32. Calif
  33. Cassandra
  34. Digimarc Marcspider/CGI
  35. Checkbot
  36. ChristCrawler.com
  37. churl
  38. cIeNcIaFiCcIoN.nEt
  39. CMC/0.01
  40. Collective
  41. Combine System
  42. Conceptbot
  43. ConfuzzledBot
  44. CoolBot

Popular posts from this blog

faq

Frequently Asked Questions This is a list with frequently asked questions about web robots. Select the question to go to the answer page, or select on the eye icon after the question to show the answer in this page. About WWW robots What is a WWW robot? What is an agent? What is a search engine? What kinds of robots are there? So what are Robots, Spiders, Web Crawlers, Worms, Ants? Aren't robots bad for the web? Are there any robot books? Where do I find out more about robots? Indexing Robots How does a robot decide where to visit? How does an indexing robot decide what to index? How do I register my page with a robot? How do I get the best listing in search engines? For Server Administrators How do I know if I've been visited by a robot? I've been visited by a robot! Now what? A robot is traversing my whole site too fast! How do I keep a robot off my server? Robots exclusion standard Why do I find entries for /robots.txt in my l...
Read more

other

Other Sites Google Many people end up on this site because they have questions about specific search engine robots and search engines. For such questions the best place is the relevant's site's own help pages: Google Web Search Help Center Google Webmaster Help Center Yahoo!'s Web Crawler Help Pages Extensions to the Robots Exclusion Protocol Recently three major search engines have collaborated to support extensions to the /robotst.txt directives and related mechanisms. See the join announcements on: Yahoo! Search Blog Google Webmaster Central Blog Microsoft Live Search Webmaster Team Blog Sites about Search Engines Other sites that are useful to webmasters: Search Engine Land Search Engine Watch Search Engine Roundtable
Read more