HS Search module showing duplicate blog posts.

Develop, Build & Integrate CMS Development
1

During the development of one of our sites we used a test blog with real blog posts and we called it “Test Blog”. When we launched we either changed the name to “Blog” or created a new blog and removed “Test Blog”. When you search for certain blog posts, it will return duplicate posts, one with /blog/ in the url and one with /test-blog/ url that goes to a 404.
There is no instance of “Test Blog” anymore and there are no re-directs still in place. I have this to robots.txt but it still returns when you search:

Disallow: /test-blog/
Disallow: /test-blog/*

What else can I try?

2

Hi, @MHowe :waving_hand: Thanks for your question.

I have a few of my own to help give the community more context:

  • Are these old URLs are still indexed by Google or another search engine? If so, have you already requested a re-crawl or removal of the obsolete URLs?
  • Are these duplicates appearing in the HubSpot CMS search or on an external search engine?
  • Is there any chance that “test-blog” URLs might have been hardcoded anywhere on your site? For example, in blog templates, modules, or elsewhere?

Thank you for the additional information! — Jaycee

3

Thank you for responding Jaycee:

  • I will look further into this, I am not 100% sure.
  • Just the HubSpot CMS, more specificaly the native HubSpot search module. When I search in Google only one listing (the correct one) comes up)
  • There are no hard-coded urls that would include “test-blog”