# This disallows all bots to several directories and files:
User-agent: *
Allow: /