Social Media Content Preview Bots to Whitelist in Robots.txt

Social media platforms like Facebook, X (Twitter), and WhatsApp use crawler bots to fetch your content’s title, image, and other details to show nice previews when someone shares your link. If you block these crawlers, your links might look broken or plain. It’s usually best to allow them, so check that your site’s robots.txt file doesn’t block them.

whatsapp-preview.png

Robots.txt:

# Allow social media crawlers

User-agent: Twitterbot
Disallow:

User-agent: facebookexternalhit
Disallow:

User-agent: LinkedInBot
Disallow:

User-agent: Pinterestbot
Disallow:

User-agent: WhatsApp
Disallow:

User-agent: Slackbot
Disallow:

User-agent: Discordbot
Disallow:

User-agent: TelegramBot
Disallow:

User-agent: Snapchat
Disallow:
Was this article helpful?
0 out of 0 found this helpful