# Whitelist Meta Crawlers User-agent: facebookexternalhit Allow: / User-agent: Meta-ExternalAgent Allow: / # Rules for all other robots User-agent: * Allow: /