Robots.txt not updating

PCHamp
Member

Hi, I'm updating the robots.txt file, but the changes aren't showing up on the live page. I still see the old file. Any idea why this might be happening?

0 Upvotes
1 Accepted solution
Josh
Solution
Recognized Expert | Diamond Partner
Recognized Expert | Diamond Partner

Hi @PCHamp,

 

This is potentially just a caching delay. If you've already cleared the cache on your browser and it's still happening, it could be a server-side cache which would fix itself (but should have by now). 

 

If that still doesn't do the trick, just double-check that you've uploaded the correct robots.txt file. When file names are the same, it's easy to grab the wrong version of the file.

 

For reference, just in case: https://knowledge.hubspot.com/social/configure-your-websites-robots.txt-and-image-meta-tags-for-soci...

 

Josh

 

 

 


Did this post help solve your problem? If so, please mark it as a solution.

Josh Curcio

HubSpot support and inbound marketing for OEMs, contract manufacturers, and industrial suppliers.
HubSpot Diamond Partner & HubSpot Certified Trainer

View solution in original post

0 Upvotes
2 Replies 2
Josh
Solution
Recognized Expert | Diamond Partner
Recognized Expert | Diamond Partner

Hi @PCHamp,

 

This is potentially just a caching delay. If you've already cleared the cache on your browser and it's still happening, it could be a server-side cache which would fix itself (but should have by now). 

 

If that still doesn't do the trick, just double-check that you've uploaded the correct robots.txt file. When file names are the same, it's easy to grab the wrong version of the file.

 

For reference, just in case: https://knowledge.hubspot.com/social/configure-your-websites-robots.txt-and-image-meta-tags-for-soci...

 

Josh

 

 

 


Did this post help solve your problem? If so, please mark it as a solution.

Josh Curcio

HubSpot support and inbound marketing for OEMs, contract manufacturers, and industrial suppliers.
HubSpot Diamond Partner & HubSpot Certified Trainer

0 Upvotes
PCHamp
Member

Hi Josh, thanks for getting back to me. I initially suspected it might be the Cloudflare cache, so I waited for a while. I updated the robots.txt file through Settings > Content > Pages > SEO & Crawlers. Is there anything else I need to do to ensure the changes take effect?

0 Upvotes