# /robotos.txt file for http://www.crsid.com User-agent: * Disallow: / User-agent: ia_archiver Disallow: /