#Disallow urls User-agent: * Disallow: /admin #Specific crawlers User-agent: Adsbot-Google User-agent: Mediapartners-Google