User-agent: * Disallow: /cgi-bin/bol/calendar/protected/ User-agent: * Crawl-delay: 20 User-agent: Googlebot Crawl-delay: 20 User-agent: magpie-crawler Crawl-delay: 20 User-agent: Baiduspider Crawl-delay: 20 User-agent: YandexBot Crawl-delay: 20