User-agent: * Disallow: /user.php User-agent: AhrefsSiteAudit Allow: / User-agent: AhrefsBot Allow: /