# robots.txt file # # purpose to World Wide Web Robots trawling your web site (or parts thereof) # # For more info consult: # http://info.webcrawler.com/mak/projects/robots/norobots.html User-Agent: Googlebot-Image Disallow: / User-agent: * Disallow: /*.xls$ User-agent: Googlebot Disallow: /*.gif$ User-agent: Googlebot Disallow: /*.jpg$ User-agent: Googlebot Disallow: /*.png$ User-agent: * Disallow: /files/manager Disallow: /filess/assets Disallow: /picture_library Disallow: /plesk-stat User-agent: Slurp Crawl-delay: 1800 User-agent: YahooSeeker Crawl-delay: 1800 User-agent: Yahoo-MMCrawler Disallow: / Crawl-delay: 1800 User-Agent: OmniExplorer_Bot Disallow: / User-Agent: gigabot Disallow: / User-agent: ia_archiver Disallow: / User-agent: IRLbot Disallow: / User-agent: MSNBot-Media Disallow: / User-agent: MSNBot-NewsBlogs Disallow: / User-agent: MSNBot-Products Disallow: / User-agent: MSNBot-Academic Disallow: /