June 2025posted on 06.16.2025X-Robots-Tag The HTTP Header X-Robots-Tag is used to instruct search engines on how it should handle indexing the page. It takes in any rule that can be specified in the robots meta tag (<meta name="robots" content="noindex">). There is a whole list of values that can be used: https://http.dev/x-robots-tag. I noticed it being used on Netlify preview URLs (example): X-Robots-Tag: noindex Which makes sense because these are draft URLs that should not be indexed.No reactions yet
X-Robots-Tag
The HTTP Header
X-Robots-Tagis used to instruct search engines on how it should handle indexing the page. It takes in any rule that can be specified in the robotsmetatag (<meta name="robots" content="noindex">).There is a whole list of values that can be used: https://http.dev/x-robots-tag.
I noticed it being used on Netlify preview URLs (example):
Which makes sense because these are draft URLs that should not be indexed.