Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp)
link_checker/1.1
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
msnbot/1.0 (+http://search.msn.com/msnbot.htm)
Gigabot/2.0att
IRLbot/3.0 (compatible; MSIE 6.0; http://irl.cs.tamu.edu/crawler)
ia_archiver
Snapbot/1.0 (Snap Shots, +http://www.snap.com)
boitho.com-dc/0.86 ( http://www.boitho.com/dcbot.html )
boitho.com-dc/0.85 ( http://www.boitho.com/dcbot.html )
msnbot-media/1.0 (+http://search.msn.com/msnbot.htm)
Mozilla/5.0 (compatible; SnapPreviewBot; en-US; rv:1.8.0.9) Gecko/20061206 Firefox/1.5.0.9
Mozilla/5.0 (Twiceler-0.9 http://www.cuill.com/twiceler/robot.html)
msie 6.0
boitho.com-dc/0.82 ( http://www.boitho.com/dcbot.html )
Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1) VoilaBot BETA 1.2 (http://www.voila.com/)
Speedy Spider (http://www.entireweb.com/about/search_tech/speedyspider/)
Mozilla/5.0 (compatible; YodaoBot/1.0; http://www.yodao.com/help/webmaster/spider/; )
Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot)
Accoona-AI-Agent/1.1.2 (aicrawler at accoonabot dot com)
ShopWiki/1.0 ( +http://www.shopwiki.com/wiki/Help:Bot)
Sogou Push Spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)
Gigabot/3.0 (http://www.gigablast.com/spider.html)
MJ12bot/v1.2.0 (http://majestic12.co.uk/bot.php?+)
Sogou Orion spider/3.0(+http://www.sogou.com/docs/help/webmasters.htm#07)
Mozilla/5.0 (compatible; Exabot Test/3.0 (Gilles); +http://www.exabot.com/go/robot)
Mozilla/5.0 (SnapPreviewBot) Gecko/20061206 Firefox/1.5.0.9
bot/1.0 (bot; http://; bot@bot.bot)
VisBot/2.0 (Visvo.com Crawler; http://www.visvo.com/bot.html; bot@visvo.com)
CazoodleBot/CazoodleBot-0.1 (CazoodleBot Crawler; http://www.cazoodle.com/cazoodlebot; cazoodlebot@cazoodle.com)
Mozilla/5.0 (compatible; archive.org_bot/1.13.1x +http://crawler.archive.org)
Mozilla/5.0 (compatible; Yahoo! Slurp China; http://misc.yahoo.com.cn/help.html)
Gigabot/2.0/gigablast.com/spider.html
VadixBot
FrancesurfBot (http://www.francesurf.net)
Mozilla/5.0 (compatible; Exabot Test/3.0; +http://www.exabot.com/go/robot)
woriobot (+http://www.worio.com)
noxtrumbot/1.0 (crawler@noxtrum.com)
Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (like Gecko) (Exabot-Thumbnails)
EasyDL/3.04  http://keywen.com/Encyclopedia/Bot
Googlebot-Image/1.0
Yahoo-MMCrawler/3.x (mms dash mmcrawler dash support at yahoo dash inc dot com)
googlebot/2.1 (+http://www.google.com/bot.html)
Mozilla/5.0 (compatible;YodaoBot-Image/1.0;http://www.yodao.com/help/webmaster/spider/;)
Yeti/0.01 (nhn/1noon, yetibot@naver.com, check robots.txt daily and follow it)
HMSE_Robot
psbot/0.1 (+http://www.picsearch.com/bot.html)
Gigabot/2.0 (http://www.gigablast.com/spider.html)
GigaBot/1.0
Gaisbot/3.0+(robot06@gais.cs.ccu.edu.tw;+http://gais.cs.ccu.edu.tw/robot.php)
MSRBOT (http://research.microsoft.com/research/sv/msrbot/
Mozilla/4.0 (compatible; NaverBot/I/1.0; http://help.naver.com/delete_main.asp)
Mozilla/5.0 (compatible;archive.org_bot/heritrix-1.13x +http://bibnum.bnf.fr/robot/)
Mozilla/5.0 (compatible; askpeter_bot/3.2; +http://www.askpeter.info)
Mozilla/5.0 (compatible; MJ12bot/v1.2.1; http://www.majestic12.co.uk/bot.php?+)
MSNBOT_Mobile Mozilla/2.0 (compatible; MSIE 4.02; Windows CE; Default)
Mozilla/4.0 (compatible; NaverBot/1.0; http://help.naver.com/delete_main.asp)
DealGates Bot/1.1 & Michalski Network (http://www.dealgates.com/bot.html)
CazoodleBot/Nutch-0.9-dev (CazoodleBot Crawler; http://www.cazoodle.com; mqbot@cazoodle.com)
Grub/2.0 (Grub.org crawler; http://www.grub.org/; bot@grub.org)
Mozilla/5.0 (superbot.com; +http://www.super.info)
MJ12bot/v1.0.8 (http://majestic12.co.uk/bot.php?+)
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; QihooBot 1.0  qihoobot@qihoo.net)
Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Wanadoo 6.2; MisterBot; .NET CLR 1.0.3705; Media Center PC 3.1; .NET CLR 1.1.4322)
ArabyBot (compatible; Mozilla/5.0; GoogleBot; FAST Crawler 6.4; http://www.araby.com;)
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1; maxamine.com-robot) Gecko/20061010 Firefox/2.0
Robotgenius crawler/Nutch-1.0-dev (http://robotgenius.net; misc at robotgenius dot net)
Mozilla/5.0 (compatible; Jeanie_2008_bot/3.2; +http://www.quickengines.com)
msnbot/1.1 (+http://search.msn.com/msnbot.htm)
WikioImagesBot 1.0
EnaBot/1.1 (http://www.enaball.com/crawler.html)
TurnitinBot/2.1 (http://www.turnitin.com/robot/crawlerinfo.html)
MLBot (www.metadatalabs.com)
EnaBot/1.2 (http://www.enaball.com/crawler.html)
Mozilla/5.0 (compatible; OsO; http://oso.octopodus.com/abot.html)
Mozilla/4.0 (BejiBot Crawler 1.2a)
TwengaBot/1.1 (+http://www.twenga.com/bot.html)
Mozilla/5.0 (compatible; woriobot +http://worio.com)
Mozilla/5.0 (Windows; U; Windows NT 5.1;fr;rv:1.8.1)VoilaBotCollector BETA 0.1  (http://www.voila.com/)
IIITBOT/1.1 (Indian Language Web Search Engine; http://webkhoj.iiit.net; pvvpr at iiit dot ac dot in)
mtobot/(+http://www.meteou.eu/moteur/)
PhpDig/1.8.8 (+http://www.phpdig.net/robot.php)
Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1) VoilaBot BETA 1.2 (support.voilabot@orange-ftgroup.com)
IRLbot/3.0 (compatible; MSIE 6.0; http://irl.cs.tamu.edu/crawler/)
Mozilla/5.0 (compatible; discobot/1.0; +http://discoveryengine.com/discobot.html)
BigBellyBot v1.0
MLBot (www.metadatalabs.com/mlbot)
Mozilla/5.0 (compatible; Exabot-Images/3.0; +http://www.exabot.com/go/robot)
Acoon-Robot 4.0.2.17 (http://www.acoon.de)
Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 4.0; obot)
CCBot/1.0 (+http://www.commoncrawl.org/bot.html)
Yeti/1.0 (+http://help.naver.com/robots/)